| 00:10 | Okay, I think it is And so welcome, everybody. And |
|
| 00:20 | am but Johnson. And hopefully you see me at this point on your |
|
| 00:28 | . And I've seen teaching this course few times by now, but not |
|
| 00:35 | . I took over the course a years. Well, 34 years back |
|
| 00:42 | some other faculty members that do You has an interest for long time, |
|
| 00:50 | fact, and America competitions. So topic in itself is something that I'm |
|
| 00:57 | in I'm familiar with. Otherwise, , as someone else, my focus |
|
| 01:06 | shifted from pure in American algorithms to computer architectures software and in particular, |
|
| 01:13 | efficiency and computing that every now and I will make remarks about, |
|
| 01:21 | how it relates to some of the in this particular course that I will |
|
| 01:29 | about today. But before I I like also to introduce the |
|
| 01:34 | So, yes, Bakshi, that hopefully there and maybe slash. You |
|
| 01:41 | comment a little bit about background and . Uh, everyone. I'm |
|
| 01:49 | but she e a PhD student working Dr Johnson. My research focuses in |
|
| 01:56 | performance and energy efficient computing. Andre mentioned. I'll be your t A |
|
| 02:02 | the semester. So I look forward the opportunity on mhm. Yeah, |
|
| 02:08 | I remember correctly, this is the time you are too young in this |
|
| 02:14 | . Yes, yes, that's So, yeah, it's just a |
|
| 02:18 | bit experience from assignments and helping answering that you may have. So the |
|
| 02:26 | I structured this'll lecture is I spent first 10, 15 or so minutes |
|
| 02:33 | a little bit about but, encompassed in the course and how is |
|
| 02:41 | of structured or organized. And um, once I've answering the questions |
|
| 02:49 | those high level topics, I will to talk about the subject of the |
|
| 02:59 | . So the first hole part here the first, you know, |
|
| 03:03 | 15 or so minutes this first have few slides, try to motivate |
|
| 03:11 | you, in case you're not already motivated to take this particular course and |
|
| 03:17 | you should take it and what is for and then talk more specifically |
|
| 03:25 | of course, and the topics covered practical matters on the finally an outline |
|
| 03:32 | the schedule of the class as we it now, and I guess I |
|
| 03:38 | also say that place a free raise your hand or used to chat |
|
| 03:46 | the way we tried to do these that I tend to focus on the |
|
| 03:53 | I'm supposed to give you and suggestions monitoring that. I do pay attention |
|
| 04:02 | , you know, race hands, question that your behalf with chats that |
|
| 04:05 | may generate. So so please feel as you have things you want to |
|
| 04:14 | about, right? All right. the first, then moving on their |
|
| 04:19 | , try to motivate why, Where topics of the course plays a role |
|
| 04:25 | much everywhere, I would say. I choose to use and timely topic |
|
| 04:31 | terms off the virus and try to it. Onda how Thio create |
|
| 04:38 | And this is just one example that problem and, you know, showed |
|
| 04:44 | in the press a couple of months . Um, actually, a few |
|
| 04:49 | ago now it's already beginning of the . So this was done in the |
|
| 04:53 | for summer. Um, computational molecular , um, analyzed a lot of |
|
| 05:03 | . So it's It's part of both analytics. One of this case particular |
|
| 05:09 | molecular science another one sort of in more or less the same area in |
|
| 05:15 | of health care and being able to large amounts of data. In that |
|
| 05:20 | , for doing diagnostics more than discovering for molecules and how they behave other |
|
| 05:30 | in terms of, you know, Analytics, it's a popular topics. |
|
| 05:33 | why I started to show some examples Data Analytics depends on numerical methods and |
|
| 05:41 | how they work and just in terms monitoring things. So then there's a |
|
| 05:47 | of some of it Israel time, some of it is close to real |
|
| 05:52 | , like monitoring what happens, for , in terms of aircraft and another |
|
| 05:59 | . That's what kind of things. an old technology enough particularly tech |
|
| 06:05 | It's actually is getting there to even the slow moving freight trains that do |
|
| 06:10 | lot of analysis and have lots of on the train to prevent breakdowns in |
|
| 06:15 | case, more than try to optimize aspects of great another one that is |
|
| 06:23 | . Thio sort of, perhaps normal is in terms off green energy |
|
| 06:30 | in this case, windmills. So that is, uh, in terms |
|
| 06:36 | the placement. Where to put There's a lot of analysis of whether |
|
| 06:40 | to try to figure out where the are good for placing when bills, |
|
| 06:45 | addition to them later on. Boring . Working on the design of turbines |
|
| 06:51 | well as the blades and all of rest of it. This part. |
|
| 06:57 | they both day analytics as well as or traditional engineering designer turbines and blades |
|
| 07:06 | this is a little bit to the off, sort of pointing out the |
|
| 07:09 | and General Electric is that is a that I'm sure many of you have |
|
| 07:15 | about to do all kinds of And there are. They made an |
|
| 07:20 | that just sort of improving things by . In terms of efficiency, it |
|
| 07:25 | be worth it about 300 year. it is, um, good Thio |
|
| 07:31 | these things other things closer to computer or computing as we know what as |
|
| 07:38 | graphics on that use good numerical methods order to sometimes simulate physical phenomena, |
|
| 07:48 | of us you search pretty much on daily basis. That's also depends on |
|
| 07:52 | miracle methods, and we'll talk about of these methods towards the end of |
|
| 07:57 | glass. They're somewhat more advanced than beginning part and all kinds of |
|
| 08:03 | Now that one does both for, , south driving cars or assisted driving |
|
| 08:11 | as well as Jones and many And as we know, even on |
|
| 08:18 | , I got your food delivered if we were on campus Yeah, Mobile |
|
| 08:25 | . And then, of course, in Houston both on the engineering |
|
| 08:30 | uh, in this case in terms seismic processing, trying to figure out |
|
| 08:36 | they're interesting parts under the surface where might be good sources of energy or |
|
| 08:45 | coming back to the health care of science in terms off doing molecular |
|
| 08:51 | which is kind of the top sequence images on this slide. Um, |
|
| 08:57 | depends on very efficient American methods to what you see as the second image |
|
| 09:03 | the left hand side that looks like mostly in always. But this is |
|
| 09:07 | of the starting point for molecular and clearly with this icing low signal |
|
| 09:15 | noise racial, it depends on very computation. American efforts to get to |
|
| 09:23 | result and There's kind of more traditional design, um, problems, that |
|
| 09:32 | dynamics in particular that uses advanced. this is another one from jet engine |
|
| 09:40 | . It also depends on these And I put this towards the end |
|
| 09:46 | that's another started all the data That is, um, I have |
|
| 09:51 | for jobs later on, for it's big and expanding type of need and |
|
| 09:59 | government and industry and commerce. But also has a different aspect of |
|
| 10:07 | That is, it's computational, very , and that as, um made |
|
| 10:18 | , uh, pay a lot of to data representation and the numerical methods |
|
| 10:25 | used to do it in terms of different forms of machine learning algorithms. |
|
| 10:32 | another aspect of that is that traditional , you know by interim D or |
|
| 10:43 | are not ideally suited to these So that means the companies like |
|
| 10:52 | Amazon, Microsoft, are becoming increasingly of vertical companies. So they I |
|
| 11:00 | all engaged in designing the even their hardware. And in that design, |
|
| 11:07 | representation also plays a big role. , so this we will not talk |
|
| 11:15 | harbor design in this course, But just wanted to point out that data |
|
| 11:22 | that we will talk about even today is a big element in how to |
|
| 11:28 | things sufficient. And there is just example of things that may not think |
|
| 11:35 | in general because most researchers never have face the bill, are thinking about |
|
| 11:44 | much what happens behind no keyboard in off energy consumption. But this is |
|
| 11:52 | of very since study that showed the country energy consumption in or indirectly. |
|
| 12:00 | it does shows, UH, the emissions, or C 02 emissions, |
|
| 12:06 | the U. S energy mix of , gas and renewables and figuring out |
|
| 12:12 | emissions for different types of activities. as it shows on this slide to |
|
| 12:19 | the training for this particular transformer type neural architectures search, yeah, consumes |
|
| 12:26 | much energy for doing one off these as the lifetime off five cars. |
|
| 12:36 | it's nontrivial expense as well as a impact, and this is increasingly becoming |
|
| 12:41 | concern in designing machine learning algorithms. anyway, this is kind of what |
|
| 12:48 | had this a back on to, will hopefully keep you're interested in taking |
|
| 12:55 | course and where it's useful. And I was trying just specify or describe |
|
| 13:03 | little bit more what the focus of numerical methods are. And as far |
|
| 13:08 | I'm concerned, it's all about the in many different ways. On the |
|
| 13:16 | thing that we're talking about today is approximation. All representation of data, |
|
| 13:22 | values. We used digital computers. things are being describe tried things that |
|
| 13:30 | numbers instead of analog numbers as he kind of a continuous range. Eso |
|
| 13:38 | one of the issues that comes in terms of approximations. And then, |
|
| 13:43 | course, there different methods that being on computing using those value representations or |
|
| 13:54 | . Andi. Much of the Well, this guy, if you |
|
| 14:01 | mostly the very basic ones that they encounter first but then focus a lot |
|
| 14:08 | trying to understand their behavior and what can expect in terms off accuracy, |
|
| 14:18 | , convergence, if you have some of interested procedure and from the computing |
|
| 14:25 | again referring to the machine learning example about how much work or how |
|
| 14:31 | what is the arithmetic? Complexity? are their memory requirements? So a |
|
| 14:37 | of it to try to understand and able to interpret the results that you |
|
| 14:44 | from using common numerical methods. Most after you take this course. And |
|
| 14:51 | in your course, you will use mathematical software, which there's plenty out |
|
| 15:00 | . But that does not necessarily tells those Softwares Uh huh much you can |
|
| 15:10 | the result. The number comes and if it that seems kind off |
|
| 15:15 | wrong than one, my tend to it's correct. That is really important |
|
| 15:22 | understand sort of the air bars associated it. Result that Jamaica some |
|
| 15:30 | Actually, they provide that. But general, so far that's not the |
|
| 15:36 | . So it's up to the individual to try to figured out how much |
|
| 15:42 | can trust the outcomes of using some these Softwares. And so this is |
|
| 15:52 | of said to give you a few methods. It's an introductory course, |
|
| 15:59 | that's the scope of it. And big part of the emphasis is to |
|
| 16:07 | how they behave in what you can and have some sense or accuracy at |
|
| 16:15 | and a little bit also about the memory requirements of different algorithms, we'll |
|
| 16:24 | math lab for lots of the Matt Lab is the most commonly used |
|
| 16:35 | for, and it's a very rich on tools is probably the best way |
|
| 16:41 | describe it, and many of you have used meth lab in various |
|
| 16:49 | It has lots of features also for thoughts tables, which is usefully making |
|
| 16:59 | . There are other alternatives, but of them are as widely distributed as |
|
| 17:05 | lamb, which is the reason why , like most others, teach this |
|
| 17:15 | in many other universities. Use map , and it's most likely the sort |
|
| 17:24 | basic software that you will find work after leaving the university if you're going |
|
| 17:33 | industry or even academia. So that's I think it's also for those of |
|
| 17:39 | who don't already know my flab. a good opportunity to get familiar with |
|
| 17:45 | stuff, and that says on the slide, it's something that you |
|
| 17:52 | uh, can download it. Licensed the university suits free for you to |
|
| 18:01 | and maps. Lamb is in originated by some academics and the way |
|
| 18:11 | company networks that produces meth lab. work a lot with academics to a |
|
| 18:17 | off innovations in numerical methods get, , integrated Internet because the close collaboration |
|
| 18:26 | the company has been many academics. it is also very much state of |
|
| 18:33 | art in terms of our using American . So, uh, this is |
|
| 18:43 | list of topics. And now talk a couple of war slides here, |
|
| 18:48 | then I will stop and take Eso feel free to think about what |
|
| 18:54 | may want to ask about, but do a couple more slides before I |
|
| 19:01 | . So this is just a list topics of today. After going through |
|
| 19:05 | general aspects of the course, I'll about 14 point numbers. Then, |
|
| 19:14 | , not next lecture next lecture. want to talk about that lab, |
|
| 19:18 | those of you are not familiar with Flateman, even for your the rest |
|
| 19:22 | you. But in particular for those familiar with match, I'm not |
|
| 19:26 | Hopefully be useful. Then move on the miracle topics, and there's the |
|
| 19:35 | seriously expansions. And then those particular expansions is used from almost every chapter |
|
| 19:42 | the book or every topic that will covered. It's going forward and in |
|
| 19:52 | to understand again the behavior on the methods being used, then hopefully it |
|
| 20:04 | be refresher. And for most of guys, an elimination, the standard |
|
| 20:08 | you're solving, then their systems of that, um and after that we'll |
|
| 20:18 | a little bit about nonlinear equations in wrote. Finding so and in particular |
|
| 20:28 | a very common method that someone you be familiar with. Calmly trans method |
|
| 20:34 | is is not only for solving nonlinear of equations for, you know, |
|
| 20:41 | forms of models that may be but it's also something used in computing |
|
| 20:46 | and square roots that comes as Part off language is basically built in |
|
| 20:57 | in many language definitions, and that's that completely scientist they have to |
|
| 21:03 | Depending on formation that working for some doing you compilers or people building |
|
| 21:14 | I'm gonna talk about the interpolation differentiation and integration. Americal differentiation is |
|
| 21:22 | very tricky business. In fact, tend to, uh, amplify |
|
| 21:31 | So whereas integrations as smoking effects, we'll talk about that and then spines |
|
| 21:41 | are widely used in computer, graphics other forms off approximations to get smooth |
|
| 21:50 | on functions or surfaces for dynamical systems things that evolves in time in various |
|
| 21:59 | . Ordinary differential equations. Houston will some of that in this course. |
|
| 22:04 | will not talked about what's known as differential equations in this course, but |
|
| 22:13 | differential questions that a good starting It's not just the partial differential equations |
|
| 22:21 | a little bit of the basic number methods, including very basic versions or |
|
| 22:26 | they want to see. Clinton Canadian that is very efficient software that is |
|
| 22:35 | used for large systems aan den. the end, we'll talk about taking |
|
| 22:44 | , or something called single divided. composition that is one of the key |
|
| 22:50 | off. For instance, search That two page rank algorithm Google uses |
|
| 22:58 | shouldn't be able. Thio hopefully talk little bit, but the Squares methods |
|
| 23:03 | kind of a fitting method than you more data than parameters that you want |
|
| 23:10 | determined some stuff that's good. Check out Hard to solve over determined systems |
|
| 23:17 | equations and a good way, and time permits. We'll also talk a |
|
| 23:25 | bit about what's known as fast Fourier it is probably one off the most |
|
| 23:34 | used algorithms in any settings and signal engineering design. You mentioned it when |
|
| 23:43 | think of it, and somewhere in very well may be for your transforms |
|
| 23:49 | the form of fast Fourier transform involved finally, something that is in |
|
| 23:54 | tricky and will not talk about all trickery. But if there is time |
|
| 24:03 | to alert to the pitfalls in random generators out there to make you aware |
|
| 24:10 | being very careful about what random number you might producing, it should be |
|
| 24:17 | lost. Yeah, and some other matters. The textbook ISS This book |
|
| 24:26 | in Kincaid on that island many have used is very commonly used at the |
|
| 24:35 | recent edition is It's not all that , but it's the topics are very |
|
| 24:43 | in the course I'm so I haven't found anyone that is better so that |
|
| 24:51 | I'm still using, even though look it and, you know, try |
|
| 24:56 | evaluate whether it's something newer that makes sense to use. You can get |
|
| 25:06 | hard copies of it, and there's various forms off kind of renting or |
|
| 25:12 | electronic versions on the boat. Do expect you to be familiar with the |
|
| 25:20 | of the book? So I encourage Thio find some version of it so |
|
| 25:29 | can actually read and study beyond what's in the lectures and the slides and |
|
| 25:37 | . That type of upload. There several versions of these books that has |
|
| 25:47 | a little bit. I do not all the differences between different versions of |
|
| 25:57 | book, but the changes it's often quite minor. So even if you |
|
| 26:03 | kept seventh Edition, most likely it be okay. The model them for |
|
| 26:14 | is so Homework's not out of the . That is what you're saying, |
|
| 26:18 | ? I will use mostly examples and textbook for assignment. But sometimes I |
|
| 26:28 | them a little bit. But the really comes from the textbook. |
|
| 26:36 | thanks. Okay, so, um yes, I started to say |
|
| 26:44 | The assignments at all try toe have assignment per chapter. And I think |
|
| 26:54 | first few chapters should be relatively easy digest. Whereas the later chapters problem |
|
| 27:05 | harder. Thio digest for you So that means that the beginning, |
|
| 27:14 | , we go through the chapters, assembly quickly, maybe a week or |
|
| 27:22 | to, um, to three lectures chapter. And then there will |
|
| 27:27 | um, an assignment on then. this will kind of the in the |
|
| 27:34 | , more or less up until spring . And then I think right now |
|
| 27:40 | tohave three assignments after the spring There are lots off material on math |
|
| 27:54 | on the Web, so you can get that so ideas and help for |
|
| 28:01 | to use my flab any other aspect my family's. Given that it's a |
|
| 28:09 | use that is also they're both That's books, um, outside itself as |
|
| 28:15 | as necessary. Lots of material on Web. Mhm, um, |
|
| 28:24 | I post on Blackthorn, and right you have slides from we've asked |
|
| 28:36 | Um, all right, taught this and slightly updated slides for today's |
|
| 28:50 | so I don't expect many changes, that's why I kind of left or |
|
| 28:56 | available slides from last year off the because they're elements. If you want |
|
| 29:06 | look ahead and see what's coming, may be useful that things may again |
|
| 29:15 | a little bit, but I don't big changes. I will also upload |
|
| 29:24 | . Uh, from the lectures after lecture will only take an hour or |
|
| 29:30 | . Uh, the way things works that Zoom does the court video. |
|
| 29:43 | so the automatic eyes without, um . So, yes. Did you |
|
| 29:52 | that it actually gets recorded because they supposed to be automatically recording? |
|
| 29:58 | It says recording on top. All right. S Oh, |
|
| 30:04 | I started to say so. It about for the kind of file format |
|
| 30:12 | zoom used to be converted and before is the format off the video that |
|
| 30:20 | be uploaded on the radio concert or university or report? I don't know |
|
| 30:27 | one of them have decided. They want videos on blackboard. So but |
|
| 30:33 | video points that or website is supported by the college. So I use |
|
| 30:43 | to upload the videos of the but it may take an hour or |
|
| 30:46 | before it shows up after class, the midterm in a final exam and |
|
| 30:55 | are kind of time. And it happen in the middle. The term |
|
| 31:01 | the lecture hours for thanks now scheduled the first week after spring break, |
|
| 31:12 | the final exam is scheduled by the . And right now I have. |
|
| 31:17 | at this point, University has decided be the final exam time for the |
|
| 31:23 | . But it happens that before it happens, the university changes to schedule |
|
| 31:30 | final exams. So you're encouraged to this check to university website. That's |
|
| 31:36 | only way I get information to for the final exam is supposed to take |
|
| 31:46 | . Um, you are strongly, it says in terms of the grading |
|
| 31:52 | , to tried to keep up with in order not to fall behind. |
|
| 31:59 | otherwise it can have a dominant So that's why it's, uh, |
|
| 32:07 | not good to be late with your and, as you know, well |
|
| 32:17 | for every course of the University of this one, uh, the academic |
|
| 32:23 | applies. And for your own more than the risk of getting |
|
| 32:33 | the topic is very useful and many or particular jobs that Jimmy get after |
|
| 32:41 | class. So when you're on be familiar with the topic is really |
|
| 32:47 | , and so please do the best can and stay honest, but, |
|
| 32:59 | , assignments and exams, um, for so yes, I guess you |
|
| 33:10 | to announce. And I make myself after class on class states e post |
|
| 33:21 | officers by, uh, my And this is just a schedule of |
|
| 33:33 | . Preliminary, hopeful. Keep up it recently. Well, listen, |
|
| 33:39 | have the schedule on topics off Yeah. So you go to that |
|
| 33:48 | , I guess my general forward it I will stop and take questions before |
|
| 33:54 | to talk about the topic. look. So in a questions, |
|
| 34:06 | , I had a question S Do you have specific requirements for the |
|
| 34:14 | format lab to use because the most available one is 2020? No, |
|
| 34:19 | don't have ah, to my knowledge particular version requirement. Um, I |
|
| 34:29 | think we'll use anything that hasn't been math lab for a long time. |
|
| 34:34 | , Thank you so much. But again, it's free for download |
|
| 34:38 | the university, so and may be to get the most recent versions in |
|
| 34:43 | there has been bug fixes that there encounter about. I don't expect that's |
|
| 34:48 | said there's anything that will not be . Well, that's an older version |
|
| 34:53 | not be useful. I may have this, but do you post your |
|
| 35:07 | e past? Yeah. So slides , well, the posted on blackboard |
|
| 35:16 | the course, and videos will be on this website. Video points or |
|
| 35:23 | . That is on, I guess side, if you can see, |
|
| 35:29 | my screen now. So those will but came in. And I'm using |
|
| 35:38 | textbook. I don't supplemented with any notes. A textbook is pretty |
|
| 35:44 | and easy to read any other If not, I will switch to |
|
| 36:08 | about following point numbers, uh, to Fine. So no. |
|
| 36:22 | I will start to talk about the data representation of value representation. |
|
| 36:31 | no sudden days in the particular levels right to cover significant digits, |
|
| 36:38 | concept of what it is and And then a lot of a lot |
|
| 36:44 | it will be about floating point their properties and big falls. So |
|
| 36:53 | we do, you know, we in all walks of life. This |
|
| 37:00 | called place value notation where the value with a digit it's a function off |
|
| 37:09 | place in a string of digits. the standard model is that the mice |
|
| 37:18 | right most digit is someone was has of the least significance are and then |
|
| 37:26 | you move left the value order. the digit represents is some powers of |
|
| 37:34 | base of, well, the basis the number system. So in |
|
| 37:44 | decimal number system, the basis 10 as you move left, do |
|
| 37:52 | Uh, yeah, they are they multiplying the number with the powers of |
|
| 37:58 | according to the positions on the most position is tend to zero then |
|
| 38:04 | tend to the one etcetera. So why the that, you know |
|
| 38:11 | all this string of 1 to 3 a number of 123. So then |
|
| 38:18 | significant did just Our position is simply it's the string of digits starting from |
|
| 38:28 | left today. First that is not and then ending with the last digits |
|
| 38:36 | the right. That is correct. as we'll see, that may be |
|
| 38:42 | . Isn't they just in the But they're not always correct, So |
|
| 38:49 | significant digits are not the whole was to the number of digits that you |
|
| 38:53 | have and the impact of the significance on this position, the number of |
|
| 39:02 | did such a do you have start illustrate what this fairly trivial example that |
|
| 39:08 | from the textbook. So in this , and using Gaussian elimination even though |
|
| 39:15 | haven't talked about it, But anyone has entered the university has been taught |
|
| 39:21 | elimination before coming. Did you wait you may have for gotten it. |
|
| 39:27 | the way it works, if you forgotten, is, is that you |
|
| 39:31 | take one of these equations multiplied with suitable factor and subtracted from all the |
|
| 39:39 | questions and try to turn the coefficient front of one of the variables to |
|
| 39:45 | . So they got a system when this variable and you keep repeating that |
|
| 39:53 | you have a system with only one that you can be installed on, |
|
| 39:58 | we do something called back substitution We'll talk about that later, but |
|
| 40:02 | we're just two variables, so I'll you on this and next side. |
|
| 40:07 | kind of happens in this case if only use religious opposition and in this |
|
| 40:16 | , use a standard formula from So yes, the digits are kind |
|
| 40:23 | dropping. Uh, this for the system is fire higher. You round |
|
| 40:28 | benefits lower than trying you round So in this case, we got |
|
| 40:37 | below in the middle of the That now has three. They're small |
|
| 40:42 | , and then you can try it use a gun. Biggest contamination. |
|
| 40:46 | multiply the question of, you saying 0.1 or four x plus point |
|
| 40:53 | want to me, why equals And then you can see that you |
|
| 40:58 | that equation by to and subtracted from second equation, and then the second |
|
| 41:03 | gets coefficient zero in front of or you only have something with |
|
| 41:09 | And it turns out that to get questions second grow from the bottom of |
|
| 41:13 | slide. So now I get that wind is actually equal soon, minus |
|
| 41:19 | . 47. Now mhm. If that happens to use all the four |
|
| 41:27 | , the same procedure, I guess little bit messy and the right |
|
| 41:31 | But if you do, yes, through the same set steps. Then |
|
| 41:36 | were discovered. You get two totally value off. Why that is, |
|
| 41:41 | of being minus 5 47 it's now 3 43.9, and then the next |
|
| 41:49 | shows. If you do have 10 of positions of your ad, assume |
|
| 41:56 | now therefore values that you have all four digits to have in the original |
|
| 42:04 | are exactly correct, and they just it out with a bunch of |
|
| 42:10 | Um, to get 10 digits and use all these 10 digits and going |
|
| 42:14 | the machinery, and then you get number just different again, but not |
|
| 42:19 | different from for Tages on our 3 point something as supposed toe three for |
|
| 42:25 | terms, something so here is kind a somewhere when they exercise and showing |
|
| 42:35 | the number of there just to use your computations can have a pretty significant |
|
| 42:44 | on the results. You get, fact, that you use three digits |
|
| 42:47 | this particular case, not the singing digit in the outcome or the result |
|
| 42:53 | correct is all wrong. So this just a ministrations that the number off |
|
| 43:03 | just the position is important on It's very important to and understand how |
|
| 43:10 | significant, and that comes back to thing that I hope you will in |
|
| 43:16 | course will understanding what can go wrong using the various numerical methods. This |
|
| 43:27 | is just about the concept that you hear many times in the course about |
|
| 43:33 | at us and called, well, versus ill post. And this is |
|
| 43:40 | very fundamental and important concept in American . It's not always easy to discover |
|
| 43:49 | it's a problem is one or the on. We'll talk about that and |
|
| 43:54 | course, but the point is about . Post means that small areas or |
|
| 44:02 | disturbances in some ways to the values you work with well, not have |
|
| 44:10 | large impact on the outcome on the . So things have sort of a |
|
| 44:16 | robust in the right respect to minor . That is not true for ill |
|
| 44:24 | systems, so things can go. very sensitive to errors on. Much |
|
| 44:31 | go wrong if the problem is Unfortunately, it's so mentioned, it's |
|
| 44:40 | easy to understand or know ahead of whether the problem is in post or |
|
| 44:49 | . Okay, Mhm then, feel free if you want sense |
|
| 44:59 | ask about the topics. And obviously, using the two aspects of |
|
| 45:07 | that is important Thio be aware off errors and relative errors have enough. |
|
| 45:14 | come back to significance in a Absolute terrorist is increased. The difference |
|
| 45:22 | the true value and the approximate Ignoring the scientists, the magnitude where |
|
| 45:30 | relative is the obvious things with it be where it's just take the absolute |
|
| 45:39 | in relation to the true value. problem is, sometimes you don't really |
|
| 45:45 | the true value. If you you may not actually need to know |
|
| 45:50 | the computations because we kind of know results. So and to estimate the |
|
| 45:59 | error one use kind of the best on half off the true value, |
|
| 46:05 | may come from the approximation. It the different set of approximations. |
|
| 46:14 | and this is funny, trivial just showing that absolute terrorist, maybe |
|
| 46:23 | but relative errors can be pretty In this case, the absolute terror |
|
| 46:30 | the same for the two examples, because the true value is very different |
|
| 46:37 | one case, they relative very small the other one it's just big best |
|
| 46:43 | Algeria. It's a so it's basically . That's why in most case that |
|
| 46:51 | a relative areas probably more important than absolute pleasure, because it gives you |
|
| 46:57 | sense for again how much the results be impacted by editors. Um, |
|
| 47:12 | is just showing the effect off um, absolute and relative virus. |
|
| 47:21 | no particular magic in this case and same principle as on the previous |
|
| 47:33 | Um, sometimes I use is And we talked about things being accurate |
|
| 47:41 | a certain number of they just or . And that means how many positions |
|
| 47:50 | the string of digits that are So not just the case. When |
|
| 48:00 | use the notional that small places for other days for your numbers, |
|
| 48:13 | now the number of sensational Hopefully you've floating point the tensions before, But |
|
| 48:22 | is this is hopefully reminder, but , um, use the notion off |
|
| 48:38 | parts that will come on the next . I guess that what is known |
|
| 48:44 | an expo owner and exponents part and Manti PSA so and typically these floating |
|
| 48:54 | numbers are normalized on the top. they using base 10 and it is |
|
| 49:02 | decimal systems owners decimal point. There's integral to the left of the decimal |
|
| 49:09 | , and then it's the fraction to right of the decimal point in the |
|
| 49:17 | point. Representation. Yes, I , one has rules typically to make |
|
| 49:25 | normalized, which means there's some rules how the first part before the exponents |
|
| 49:37 | the multiplier of the base shows So as it's done on this side |
|
| 49:45 | it normally scientific notation on with the 10. The rule is that the |
|
| 49:55 | part should be zero. And for fraction the thing to the right for |
|
| 50:03 | decimal point, the Legion digit should be zero. So things yeah, |
|
| 50:13 | So like, uh, the first on the top 37 point 21 8 |
|
| 50:20 | 9 get shifted right, so to , to make the integral portion |
|
| 50:29 | But then it stops shifting it. they did, you three ends being |
|
| 50:35 | to the right to the decimal And then, of course, to |
|
| 50:38 | the correct number will need to multiplied 100 or tend to the to similar |
|
| 50:45 | the other one that is 0.2 It to be shifted less left, and |
|
| 50:52 | get shifted until the first number or , too, ends up being yes |
|
| 50:58 | the right of the decimal point. then it has to be scaled by |
|
| 51:03 | to the minus T. And formally is that symbol on this life. |
|
| 51:10 | here's again the normally scientific notation as . It's we're using the decimal number |
|
| 51:17 | and using the binary number system, is a pretty much every computer does |
|
| 51:24 | the binary system. So it's the to So it's a question. |
|
| 51:30 | Um, the powers of two in position, um, correspond to particular |
|
| 51:36 | to. So here is you can the two parts Mantis TSA and exposes |
|
| 51:43 | there. Fine. Yeah. So know, the bar chains started out |
|
| 51:52 | this lecture in there. What you saying that as we know, |
|
| 51:59 | computers tend to operate with well defined sizes. Typically 32 bits or 64 |
|
| 52:09 | . It's the most common that there's she were like, 16 8, |
|
| 52:16 | I'm calling, you know, eighties , as many of you know, |
|
| 52:20 | then there liberals around four and So is a fixed number of bits being |
|
| 52:27 | to represent entities, whatever they might , including numbers. So that means |
|
| 52:36 | takes the binary Felton point representation. can Onley represent a limited number off |
|
| 52:46 | my case, finally digits from zero once. It also means that irrational |
|
| 52:54 | cannot be represented. And, on the you know, the rational |
|
| 53:03 | that the principal could there represented correctly you had an infinite number of |
|
| 53:10 | but you don't just have a fixed , so that means there is also |
|
| 53:15 | set, rational numbers that can be correctly. And then there's this concept |
|
| 53:21 | overflowing under flow, which happens when too big to fit into the number |
|
| 53:29 | or too small Thio fit into And so examples of that in the |
|
| 53:37 | few slides. And there's also this of machine epsilon that ISS basically the |
|
| 53:48 | or distance between the smallest number stepped can be represented and also examples of |
|
| 53:59 | too. So these are important. , so the standard I will talk |
|
| 54:07 | that I Tripoli standard for representing floating numbers. They also specifications how overflow |
|
| 54:16 | under floor is supposed to the hand , and it can be detected in |
|
| 54:21 | procedures. And it's up to then software off the programmer to decide how |
|
| 54:27 | handle those situations. And again machine , and tells you what can be |
|
| 54:38 | in terms off errors. This is little bit of cartoonists illustration, or |
|
| 54:48 | is floating point representation effects the numbers you can represent, and it's very |
|
| 54:59 | so it doesn't correspond to in a standards or just using an example with |
|
| 55:06 | They're just a position and in the number system. So there's one column |
|
| 55:15 | numbers that can be represented using the minus one. The center parties Conan |
|
| 55:22 | and the right columnist for the Experiment . So that's the Exponents again tells |
|
| 55:31 | how the man Tisa escape that this first part before they multiply. |
|
| 55:42 | So the first column and it's really simple to interpret and is plotted and |
|
| 55:51 | the line below the table. Now second column is also in itself. |
|
| 55:59 | to understand, hopefully, but one to notice is that some of the |
|
| 56:04 | in the second column are already covered the first column So some of those |
|
| 56:14 | actually an alternate representation. Uh, in the first columns of, for |
|
| 56:22 | , the second role in the first ? The 0.10 times to to the |
|
| 56:27 | one or 2/16? Isn't he the as 1/8? So, in |
|
| 56:34 | the first half of the second column already covered when you first come. |
|
| 56:43 | the bottom half of the second column new representations. And, as you |
|
| 56:49 | see, then that's very obvious on line representation that the spacing between numbers |
|
| 56:56 | different because it's been the multiplier The power to is different. So |
|
| 57:02 | is the fact that it says in following point representations. That said, |
|
| 57:08 | numbers that can be represented are not spaced, and that is sometimes quite |
|
| 57:17 | , will be aware of. And also things you cannot get arbitrary close |
|
| 57:24 | the number of zero. So that's , again, things can results in |
|
| 57:29 | floor. There's numbers, small numbers came out we represent, or some |
|
| 57:36 | call it the whole That's there. , yes, I guess I should |
|
| 57:44 | out that these numbers are not in normalized representation. It was just easier |
|
| 57:52 | of the point to see how the for Montee sous risk scaled powers that |
|
| 58:00 | come out. Um, so now will talk about the I Triple E |
|
| 58:13 | point standard has been in effect, believe, since 1985 and hasn't changed |
|
| 58:25 | anything, nothing in terms off talking on the representation. They may have |
|
| 58:30 | some minor changes in terms off numbers are not to 14 point numbers, |
|
| 58:40 | again, it has mechanism, film under an overhaul and a few other |
|
| 58:49 | situations. So this is important are to be aware of, so understanding |
|
| 59:00 | floating point standard is important, so should make sure you really do you |
|
| 59:08 | how these formats works, the position can get and the range of numbers |
|
| 59:16 | in some applications. The range numbers be more important than the precision, |
|
| 59:28 | I'll come back to that because that turned out to on being the cause |
|
| 59:36 | a new format that has been introduced in the standard, but it's actually |
|
| 59:40 | supported money. Several vendors now using position, then you need causes and |
|
| 59:58 | in memory uses in performance and in consumption. So it's important to try |
|
| 60:14 | . Be judicious in your choice of , not use more than you actually |
|
| 60:21 | you need. On the other underestimating the position you need can be |
|
| 60:29 | in terms office. You're not getting accuracy that is required as so many |
|
| 60:35 | examples earlier. So on the bottom is just a big exact format that |
|
| 60:47 | specified by the standards. So all have to comply with this particular format |
|
| 60:53 | their presenting their numbers, and that codes portable. You can running on |
|
| 61:01 | platforms and the code. We should able to get the exact same result |
|
| 61:06 | the exact same data Most computers air powerful nowadays. What's the point of |
|
| 61:11 | using ah, higher precision? Uh , value. So they in terms |
|
| 61:24 | the called negative side, uh, higher position than you need. So |
|
| 61:31 | you take the example singles and double , so the double position for most |
|
| 61:39 | detectors out there give you half performance it takes twice as long to solve |
|
| 61:44 | problem. If you use double versus is single is enough. So for |
|
| 61:51 | , given were in Houston. It out much of the seismic folks. |
|
| 62:00 | find single precision good enough, so using the precision in many of their |
|
| 62:05 | , they don't give stubble. Also, for many problems you try |
|
| 62:15 | solve or focus on, or make good Jews off your memory that you |
|
| 62:24 | on and talk clusters in an old in the server. And it is |
|
| 62:30 | double position you can potentially on this has. The problem is half assed |
|
| 62:35 | as you want, and it also you to dissipate mawr energy the energy |
|
| 62:45 | retrieving information from your memory, which increasingly important given how technology scales ah |
|
| 62:58 | not exactly double but twice the number bits for a good part of the |
|
| 63:05 | also doubles that part of the energy , also the same in terms off |
|
| 63:11 | suppliers and ADDers in double position. use significantly more in double position than |
|
| 63:21 | , and I will make a few comments on that in coming, slide |
|
| 63:31 | and even coming back to my the think that's where I had the |
|
| 63:38 | that showed that effectively energy consumption for a particular very our guest. Sophisticated |
|
| 63:49 | you like. Um, machine learning model consumes as much energy as a |
|
| 64:01 | off. Five typical US courts. one training session. So if you |
|
| 64:09 | reduce that energy by using and less and still got acceptable accuracy in the |
|
| 64:21 | , that is well worth both in of money and environmentally in that help |
|
| 64:31 | the question. Yeah, that was perfect answer. Long answer. But |
|
| 64:36 | an important one. Thanks. So are, says binary, and this |
|
| 64:42 | because the basis binary, but it's point on the number after the word |
|
| 64:49 | is a number of bits being used represent. That's what's important number. |
|
| 64:59 | it used to be the most common waas, the binary 32 64 occasionally |
|
| 65:05 | some numerical methods, quadruple position or 1 28. What's necessary to get |
|
| 65:14 | results about not by no means dominating 32 64 was really common, and |
|
| 65:22 | computer graphics became more sophisticated, they ended up using binary 64 was in |
|
| 65:33 | days to use Primary 32. in recent years, um, was |
|
| 65:45 | success I should say, machine In many walks of life, Binary |
|
| 65:52 | has become much more importance of Binary 16 is also supported in |
|
| 66:04 | and the reason for that is execution and energy. This suppression in on |
|
| 66:14 | . In fact, that's a little for the uncommon to my last answer |
|
| 66:22 | energy. Dissipation is the design constraints pretty much anything today. So now |
|
| 66:33 | here's about how they're floating Point um, works and this focus on |
|
| 66:40 | ones that are most common. And that means the 32 year in |
|
| 66:44 | It's versions, um, and how here is how it's structured. So |
|
| 66:54 | the exponents, the sign of exponents implicitly represented through this so called excess |
|
| 67:05 | . So the actual exponents value you from taking the bits in the exponents |
|
| 67:12 | and then subtract the bias. And bias is well defined in the standard |
|
| 67:18 | basically as well the base. That's pirate, too, for the binary |
|
| 67:24 | the power off the number of bits one in terms of exponents on |
|
| 67:32 | So that means the biases 1 27 single on its 10 23 for double |
|
| 67:42 | of ordinary 64. The other thing be aware of. And you should |
|
| 67:48 | , um, is that the Monte the standardized to always be of the |
|
| 67:55 | months off one. And when the point and then the fraction my sends |
|
| 68:05 | leading digit is always one. According the standard, there is no point |
|
| 68:11 | representing you. So to get the value you need Thio know that this |
|
| 68:18 | is implicit soul. Look at the at the fraction to figure out what |
|
| 68:23 | means, and then you need to in the leading value of one. |
|
| 68:30 | the last are the binary partner. even though there were 33 bits for |
|
| 68:38 | Monte PSA in the binary 32 in fact you have 24 bits for |
|
| 68:47 | Mount Isa. So on. Then a couple of value said are unique |
|
| 68:57 | figure out these exceptional cases. So 24 values to the reserve for handling |
|
| 69:07 | cases s O. This is the for the single position in terms of |
|
| 69:17 | dynamic wrench of the range of numbers I don't know as well as again |
|
| 69:23 | position that you can have in the and the largest numbers. So since |
|
| 69:31 | are most reducing thinking in the decimal system, it's kind of useful to |
|
| 69:37 | off. The single position is giving about seven decimal digits off position and |
|
| 69:45 | the double position format again, about decimal digits of precision. So these |
|
| 69:57 | things expected to? No. So tend in one form or another. |
|
| 70:05 | ask these things when it comes to terms. So just making ahead. |
|
| 70:11 | pay attention to the standing because that's . And since in a piece of |
|
| 70:17 | you would use depends on it coming back to us at this notion |
|
| 70:23 | performance and energy consumption and trying to to different applications as machine learning |
|
| 70:37 | um, become popular or widely I should say, popular in that |
|
| 70:45 | and has that quite a few It's also the case that the training |
|
| 70:51 | particular is very energy and compute intensive success, many of the machine learning |
|
| 71:01 | depend sieving through and making use off , very large data sets. So |
|
| 71:14 | went ahead and decided design. It's formats or floating point number representation they |
|
| 71:30 | that the float 16 the standard I representation didn't have enough arrange. So |
|
| 71:48 | own floating point representation to be felt . Ah, they found that using |
|
| 71:57 | article e So Corbyn er, 32 . It was okay, but not |
|
| 72:06 | . 16. So they adopted a range from the Tripoli. So 32 |
|
| 72:16 | or binary 32 standard. But since only had 16 debts they ended up |
|
| 72:23 | , you were it's a position or smaller man Tisa On day one of |
|
| 72:29 | that has been exposed to machine learning learning algorithms probably seen many articles where |
|
| 72:41 | lot of the effort is trying to out how to use as few bits |
|
| 72:46 | possible in representing values. So you and for their applications discovered that in |
|
| 72:53 | case, seven bits for the Tessa was okay. So shifted the |
|
| 73:01 | between range and position. Lord, precision and increase the range compared |
|
| 73:09 | There are temporary floats, then 16 . And that was for performance |
|
| 73:18 | Time to solution, an energy And so the beef floats 16. |
|
| 73:25 | the basis for designing their own This is but it's known as the |
|
| 73:31 | , or the tensor processing unit that design from the ground up. Now |
|
| 73:39 | then, I said, is a , and it was originated by |
|
| 73:43 | But now several instructions that support it several vendors supports this particular format because |
|
| 73:51 | success in machine learning. There are some older. You know, Intel |
|
| 73:58 | what they call the extended position, an 80 bit instruction, and IBM |
|
| 74:04 | his own instructions. But the still on. I don't think there's new |
|
| 74:12 | , all that much developed rhythm. to be folks 16 definitely is something |
|
| 74:17 | is take hold in industry. so this is coming back, more |
|
| 74:26 | less to the answer that energy consumption performance is related to how many bits |
|
| 74:34 | using for your representation. And it to be that there was a big |
|
| 74:39 | in particular for mobile things and where is limited through, you know, |
|
| 74:44 | that have a limited capacity compared to that plugs interior power outlet in about |
|
| 74:52 | zealous for mentioned for over addiction now dissipation of chips, it is the |
|
| 75:01 | that limits performance or capability of the using soul producing energy dissipation is the |
|
| 75:14 | designed constructs, and that's far again , um, fewer bits for your |
|
| 75:21 | ation. And it also to get work done in the power an energy |
|
| 75:29 | that can be handled. All I had something about grounding as |
|
| 75:37 | So typically, we have this point things are number is in the |
|
| 75:44 | or, um, then there's a if you what you do. But |
|
| 75:48 | it's about the mid point, then round up on it is below your |
|
| 75:53 | of round down. That actually standard several different options for how you can |
|
| 76:01 | it surrounding and you can always run it says here to zero. You |
|
| 76:08 | run towards infinity, and you can two respondents infinity, and this little |
|
| 76:16 | shows you depending on what the the mold, what numbers you |
|
| 76:22 | And most of the systems, has ways for you in setting environment |
|
| 76:30 | to define how the running is supposed be done. Some sophisticated systems can |
|
| 76:40 | it on a per threat, as those of you are familiar with parallel |
|
| 76:45 | if anyone is. But there's where reason are doing it, but normally |
|
| 76:51 | basically done for the program as a , or even for a system that |
|
| 76:58 | is set what the rounding mode And there's again also query you can |
|
| 77:06 | for finding out what the rounding mode in the system you're using. So |
|
| 77:15 | will just stop through one of them this table, and we're encouraging to |
|
| 77:21 | at it on this case. We the values on the first line, |
|
| 77:26 | then rounded to whole digits or hold numbers. So basically no fraction. |
|
| 77:36 | I live here around the nearest and if it's time around to the |
|
| 77:42 | numbers, right, so then 11 gets number 12, but so does |
|
| 77:47 | . Because if you round up, become certain that is not the needed |
|
| 77:52 | . And, um, a bombing Similarly for the negative numbers except |
|
| 78:00 | in fact, for the 11.5 around , instead of rounding up for the |
|
| 78:05 | members So there are a number of outcomes depending on what the rounding |
|
| 78:11 | and the running can have significant impact the outcome. Ah, your |
|
| 78:21 | Um, and this one says so The standard requires that, um, |
|
| 78:32 | off the algebra operations except with sometimes the intrinsic functions, a square roots |
|
| 78:43 | what color that's the protocols needs to followed. The actual police standard was |
|
| 78:51 | like logs take functions a not required the standard toe. Follow it. |
|
| 79:03 | see, let me try to cover time all my stuff for a |
|
| 79:08 | But let's cover this and there may a couple of fights. I will |
|
| 79:12 | the next time. Yes, so just shows what can happen in terms |
|
| 79:16 | the most common way of losing Yes, when you do subtractions of |
|
| 79:21 | numbers and almost the same so sine remember the sine functions and the starch |
|
| 79:29 | zero for X being zero and then graces eventually to one what it means |
|
| 79:34 | excess close to zero, then sign more or less equals X. So |
|
| 79:41 | this case, it's a good actress evaluating sine X and attributes objections. |
|
| 79:49 | this case, the three leading it ends up all being zero, and |
|
| 79:53 | you're normalized in order to follow the , and then you get 30 shifted |
|
| 80:00 | and the end of the number and are kind of spirits. You don't |
|
| 80:05 | know if there should be zero So those I'm out. So you |
|
| 80:09 | three days opposition in this particular so on. And this is just |
|
| 80:19 | slide you can look at, but going through, you can find out |
|
| 80:24 | you can also bound the number that to lose and during this exercise and |
|
| 80:32 | . But I think I wanna have is just another example. Same thing |
|
| 80:37 | the subtraction. What happens? Let talk very quickly about this one |
|
| 80:43 | If you have questions, I will it up next time. But the |
|
| 80:48 | is that I'm trying to get for slide. It is unfortunately how you |
|
| 80:53 | this type of also significance is an . And this is one case showing |
|
| 81:00 | orchestra You need to figure out how make things better. Some of this |
|
| 81:04 | against the square root of X squared one, um is for small access |
|
| 81:11 | equal to one. So then subtraction or two numbers, that is |
|
| 81:15 | the same. So how does some with that? Well, it is |
|
| 81:21 | way best is you try to be so you know some of the rules |
|
| 81:25 | hopefully have learned that should take this on square with Mannis van and multiply |
|
| 81:33 | square root Class one. What you this. A new expression where the |
|
| 81:42 | term in the second term or each . In that case, what remains |
|
| 81:45 | basically X squared. So now they something that is in fact, more |
|
| 81:51 | than if you ask. Brute force the first expression, and here's another |
|
| 81:57 | that's that's the same thing, but again are so you treat them |
|
| 82:02 | Onda kind off. We need to . Have a good set off tools |
|
| 82:07 | your toolbox to figure out how to this expression, to get something that |
|
| 82:15 | more accurate when it comes to floating numbers and competitions. And I think |
|
| 82:23 | okay. I will stop there. sorry. Could one or two minutes |
|
| 82:28 | stuff stop on, take questions and bring up in a couple of slides |
|
| 82:35 | again get to you next time. , e guess one point I will |
|
| 82:51 | up that I forgot to bring up . So not just me and |
|
| 83:02 | trying to figure out what it's a tool for having any kind of discussion |
|
| 83:10 | between us and you and for you yourself. Many or I should |
|
| 83:18 | many, but some faculty and used is one of these things. But |
|
| 83:25 | , so changing their business model that make something several other faculty that used |
|
| 83:33 | move away from it. Some are to use Microsoft teams. But most |
|
| 83:39 | the ones are not particularly happy with platform for discussion. Question answer forums |
|
| 83:48 | one of the faculty members at and I was trying to use a |
|
| 83:52 | tool training game. So if you experiences and in particular, well, |
|
| 84:01 | positive and negative opinions about some of usefulness of these tools, please let |
|
| 84:08 | know, too, I guess. or jets? And should I mentioned |
|
| 84:15 | earlier repeated next time trying to supplement phone with something or zone with something |
|
| 84:34 | ? Yeah, it was testing a last night, and it seems pretty |
|
| 84:38 | the same as Piazza, so just little bit differently. But, |
|
| 84:43 | you are that same format. As answered, You're just supposed to have |
|
| 84:49 | and discuss it right Answer story. . So we may try it. |
|
| 84:58 | again, I encourage you're the students let us know what might serve. |
|
| 85:07 | the best. Okay, if there no questions on stop sharing this |
|
| 85:36 | I want to put their often Sure don't lose the recording. Seriously? |
|
| 85:40 | . I'll |
|
