Learning to Learn

By Richard Hamming

A lot of people worked very hard, but people who did significant things are few and were able to do it because of their style. It's matter of difference of style.

Richard says, according to history from Greek Philosophers to middle ages, there are saying that not everything can be put into words. Style is one of them.

If guy like hamming can go out and become a great scientist, I can.

You must have to do the work. There is no one successful style. You have to think carefully about what you hear and read. Discuss it with your friends and adapt to yourself. You have to take what fits you from other people and it becomes your.

You must manufacture the style which will make you a significant person in the future. It's not going to be easy.

Every future generation is going to have to deal with more difficult problems as easier once are solved by our previous generations. Now that we are already landed on moon next good feat in space is going to be a lot harder. That's definite.

Things you did to become great might not be appropriate for next generations.

Education is what, when and why we do things. Training is how to do it. You need both theory to guide you and skills to do it.

The modern era in science and engineering began with Sir Issac Newton roughly around 1642 and from there to Hamming's time around 1995 the knowledge doubled every 17 years. The 90% of scientist who ever lived are alive today.

[!todo] For the publication and journals I would have to get numbers myself..

Hamming goes on predicting the future with those number of 90% and double at 17 years with integration to see what would it look like after 340 years. At the time of newton there was only one field in science natural philosophy and after 340 years according to math it's gonna be 10 billion which isn't possible and neither all humans can be scientist. So we can see past is not too good a guide to the future.

Although what can learn form the "back of the envelop method" will help you practice quick modelling.

[!important] Rate of change is proportional to how much you have.

You have to have enough background knowledge to penetrate jargon.

[!note] Science, if you are doing it, you shouldn't know what you are doing. Because science is supposed to be the exploration of what you don't konw.

Engineering, you shouldn't be doing it unless you know what you are doing!

Develop the ideas first and then apply it is going to be less and less acceptable.

There is a saying. Short-term predictions are optimistic and long-term predictions are pessimistic.

A technological invention can change completely the history of something and one can hardly foresee technological inventions.

Social conventions are going to stop a great many things from happening. Future will be less determined by what technology can do than social, legal and other restrains on what we can do.

Knowledge is sort of homogeneous body. Which is specialized with certain names, but it's always connected together.

Don't ever try to get humans to do something reliable.

[!important] With one life to lead, you ought to do more than just get by.

Foundation of Digital discrete Revolution

It's a move from manufacturing to information society. He doesn't think robots will be run by a Von Nueumman machine but rather fuzzy logic, neural net or other things for control.

What are the fundamental things I'm trying to accomplish? How do I accomplish it with machines?

99% of the experiments are done with simulation.

More valuable things must be maintained. Putting maintenance very last thing isn't satisfactory. You must begin to plan in initial stages for the field of maintenance.

Regularly things are going to change and for him it awfully rapidly change and we know what's the stat now in 2023.

We will get to a rate of change where the populace cannot stand more rapid change. Rate of change in society will be limited to a great extent by the ability of people to keep up with change or respond. Change is painful to human beings.

Use things that help you evolve with time. Don't end up at a dead end. Don't forget analog. Combination of YES/NO/CLOSING is very powerful. Digital analog is powerful as well.

How to get reliable results out of unreliable parts? You must be responsible for your beliefs.

[!quote] I don't care where you read it. I don't care who you said it. Even if I said it, if it doesn't fit with what you believe and your common sense, then it's not so.

Buddha, 500 BC

Do I believe that guy of not? I told you he is a famous and all. But how many famous people were wrong? almost all at sometime or another. It's bound to be. What works in one generation doesn't in next. You are responsible for your decisions.

Fortran was designed psychologically and Algol was designed logically. Programmers who design logical languages don't get their language to last long. The tremendous survival of fortran tells you the value of psychological design of language as against logical.

The person who discovers something rarely understands it

The person who discovers something rarely understands it. They had their feet so far in the back and all the trouble they had they don't see the

light. Whereas the people who come afterwards see it much more clearly the person who did it.

Perhaps the best illustration of this point, besides telling you my personal story, is Newton.

It has been said Newton was the last of the ancients. He was not the first of the moderns.

Almost all creative people when they create something do this. So the reason we're telling you is twofold, if somebody else has created a field and he said this is the way to do it, you're entitled to ignore him. On the other hand, when you create a field, remember to shut up.

You don't really understand what you did. Now I started that method with four rules: easy to learn, easy to use, easy to debug, and easy to use subroutines. Now let's skip the fourth one because it's tied up with the crazy ideas we had about subroutines which are of no use to you whatsoever. It would appear that I was doing top-down programming.

Programming is closer to novel writing then it is to engineering

Because the engineer is bound by a large number of hard laws of physics and the programmer is much more bound by his mind. Any ingenious thing they can think up just like a novelist. The novelist is bound by their imagination to a great extent. The programmer is bound by their imagination much more than they are bound by hard rules. Therefore it's not so likely that you're going to be able to produce engineered software. It isn't the same problem. It's more like novel writing, as far as I can see.

Think before you write

Don't write one line of code until you have decided what your acceptance test will be.

Although there are exceptions, Some problems we don't know what the problem is and so we start programming to find out the problem and there are lot of those and I warn you about those.

Parallel machines are going to put a bigger burden on programming. We desperately need better programming.

Programmers are more than 10 to 1. Neither experience of working with large number of languages nor taking course on programming makes a good programmer. Take example of novelist. No great write comes out after taking course on creative writing. Bureaucrat who write for 20 years haven't improved their writing even after that experience. What does make great writer than? It is apparently, the ability of individual to set up a criteria of what they privately consider good programming and criticize themselves and hold themselves to a high standards of writing.

Professors who teach programming are not necessarily good programmers.

Applications of Computers

A function of a scientist is to communicate. There are written books and written articles, Formal and quick informal talks. You should master all of three. If you are going to create something there is an obligation to your part to communicate.

Informal talks

• Talking about what you are expert at isn't the great way to get invited again for a talk. Think of things they want to hear about.
• When you give talks better get profit by it also.

Richard says, Most pattern recognition problems will be done by machines. We are going to saturate polynomial type machines and have to go to parallel processing to do it.

Important decision should not be based on moment to moment data at least the long-term.

What would be the smallest program that I would consider thinking?

Indeed are you different than a machine? If we think game of chess requires thinking and we have machines that beat us at that and we say well it's just matter of formulas then what about you? do you really think? are you intelligent? by that notion chess doesn't require thinking.

If you think there a soul, then the question arises how does the soul interrupts the sequence of molecules to change what is going to happen. If you believe there is a psychic world and physical world then you must answer the question which is very vexing how do they interact? If Ideas are physical things how are they stored?

Can machines think? Real problem is can you write the program? What is a machine?

Problem of ambiguity. You both believe and disbelieve. It's quite difficult as your whole life it's been yes or no.

You can not make progress in domain of AI until you get yourself involved and ask yourself this questions.

• Can machines think?
• Can it learn from experience? What is intelligence?
• What is that fundamentally different that you have but machine don't?
• What I accept as test that machines can learn or not?
• What will I accept that there is a essential difference between man and machine beyond my own personal prejudices and religious convictions.
• Can programs be written so that they can exhibit things like learning from experience or mimic human or any other living creature.

Richard believes machines can think and can not and was struggling between the duality. Look at you are just a bunch of molecules or you are not?

You can argue that if single cell doesn't have intelligence a lot of them can't have it either. Molecules don't have friction but object does. There is a significant differnce between a individual and group. maybe thinking is like friction that emerges out of whole but parts are not having it.

are we doing things better than nature did? throughout centuries?

Brain is not a homogeneous structure.

Richard shows very interesting Geometric proof done by computer about congruence of base angles of Isosceles triangle that I've not seen in my math class or any book.

Can you produce originality? Can we simply algebraic expressions using machines?

How ambiguous are our beliefs?

He talks about the enormous variety of AI

• There are those who think that you are trying to study human beings and trying to mimic or duplicate that. It's stimulation.
• There are those who are trying to produce the result those are hard AI.

Richard thinks thinking is not what is being done but the way it's done. As he asked question previously what's the smallest program that can think and when you cut that in half it can't think anymore. But he thinks maybe it's the wrong questions. It's not a yes/no question. It's a matter of degree.

N-Dimensional Space

[!quote] Luck favors prepared mind.

How to get most out of the situation? Ask more questions. If I want to be on cutting edge and with great minds today where would I look? How would I contact them and learn from them?

You build things fro 3 dimension but you design it in N-dimensional space. So what matters in design is N dimensional space.

You think you know 3 dimensional space but you don't. You are really familiar with 2 dimensions.

As the dimensions increase the amount of volume of spear of n dimension decreases to near zero. And for dimensions around 2k It's volume is mostly on the surface. So in n dimensional design space you are looking / searching the surface.

It's better to get the right problem solved slowly than to rapidly solve the wrong problem.

What is L1, L2 and L infinity? Why is L2 not applicable to AI problems?

Greatest step in creativity is recognizing that there is a problem. Second greatest step is identifying the nature of problem.

We do not test the way you think we do by repeated trials we cannot. As a new product comes off the production line you want it in field three weeks from now. How are you going to test that from lifetime? It gets you back to the design space you would have to look at the design and look at the design space to figure out what you think the reliability is cause you can't test it.

Coding theory

What is information? Shannon created the theory of information at bell labs. Bell Labs wanted to call it communication theory but we would have to live with other word.

Source -> Encoding -> Channel -> Decoding -> Sink ↑ Noise

Information theory and coding theory assumes that there a noise to be combated. Where as in physics that was added later as you have pure mass and force but when you go to labs you realize there are other factors. Uncertainty in quantum physics is not this kind.

How can we combat noise in the channel? Transmission through space and time are the same kind of problem exactly. Shannon and Richard figured out that you can not build perfectly reliable equipment. As you you try and build more and more reliable equipment, once you get fairly well along, its enormously expensive to get further. They used differential analyzer at that time for something and he said to get one more bit accurate the cost was 10x. We are doing coding theory because we want cheap and fast as with coding theory we escape the necessity of doing things right because its very expensive to do it right. very simple but difficult fact.

Unique decodability through trees. Instantenous decodable. Kraft inequality. Damn I don't understand a single thing he's talking about Kraft inequality. Idea is not in the words

What the idea is, I don't know. I'll tell you how I measure it. If I'm teaching Integration by parts, final exams are near and that is an important topic. I would include three questions some direct and if student solves them. I will accept that they got the idea of Integration by parts. In true sense, I will never be able to know if they got the idea. There is no way of surely knowing it. We deal with Ideas but we don't know what they are. We don't know what information is. Can you write a program to recognize information or knowledge? We don't know how to recognize knowledge I can't tell you what information, ideas or knowledge is. I know what time is until you ask me and then I don't know. In the same way you always think you know what information is, what knowledge is, and what ideas are, but when you begin to press very hard, like in your business, writing a program that does it, you begin to find that you don't know what you were talking about.

Words tends to freeze something.

Error-correction codes

Before watching, I would like to find it out for myself.

If we are moving 10110101 with parity bit as 0 as Number of 1's is odd. And we get the following

10110101 -> 10010101 when calculated the parity bit to 1 and compared to 0 it doesn't match so we know there is an error of one bit.

But how do we go about finding where the error occurred without requesting anything from source as on return it will be same as sending back the block of information. Maybe bit shorter than whole sequence might help but then what should it consist of? As of now I can't think of a way we would get to know where exactly bit flip happened as with given information at Sink.

[!important]

• It is characteristics of great things that the person involved is deeply emotionally involved.
• The great things come from people who care and care passionately.
• I saw lot of great people do something great and spend their rest of their life on that thing. They did nothing creative after.
• Luck favors the prepared mind.

[!note] Two Important things

• The story you should look at mistakes and learn from mistakes is basically wrong. You should review your successes. Because when you study mistakes when your chance comes you will know how to make a mistake. If you study success when your chance comes you know how to make it a success.
• There are so many ways of being wrong as so of few of being right. It's easier to study success.

[!note] You must on your own time demonstrate greater ability and when you demonstrate that they will give you the freedom to do it but they won't by and large give you a freedom to do for

Now that I'm into the lecture somewhat, he talks about on relays maybe. You need 3 machines to check and compare. By that ideas I can think of following thing

• If we make or move more than one copy of them we can compare it. But wouldn't it be highly redundant? I don't see there is any practical benefit of it even though it might work out.

• I was unable to think what would happen if I scale it. In 8 bit sounds bad to have 3 bits to cover the cases but when it's n bits and n=10. Then total possible values are 2^10-1 which is around 1k. So for 2^10-1-n around 1013 messages can be send with 10 bit parity. For big code it's evidently better. There is possibility of double error but Hamming reserved last bit for parity of double error.

Once error correcting codes were reasonably launched. Hamming told himself not to read papers on the subject of error correcting codes. You are not going to run free papers and you aren't gonna do anything. You are going to go out and do something else.

Pythagoras was the first physicist, he found we live in L2. That the sum of the squares of the sides gives you the square of the diagonal of a rectangle. Hamming says yes, Pythagoras is right about he physical world and not about the mental world. Difference between two strings of bits is the sum of the differences. It's not the sum of the squares.

A child is coming down the street and you say well that's too tall for my daughter and that's kid isn't my daughter. That's L infinity. Only one feature is sufficient for you to make the decision. Where you say well look at this differences because of this 5 or 6 differences she isn't my daughter. Well that's L1. Sum of differences.

Information theory

How much does information theory is about information? It was developed by Shannon for telecommunication company and they wanted to have it named communication theory but to Shannon information theory sounded nice.

What is information? Shannon identified it with surprise. When something surprising happens you have learned a lot, If something less surprising happens you have learned less, If certain event happens you learn nothing.

Log of base 2. Gibbs Inequality. Noiseless coding theorem.

When definitions are made you should look carefully at distortions.

It's evident that the exam can get any distribution you want. We have used information theory and applied it to various fields to know if they have only mechanical factor or non-mechanical factors as well.

Eddington.

Instrument limits what you can think and see. Because our mind is wired in certain way.

Digital Filters

[!note] Underline If you will do a little extra work each time and get down to fundamentals and study things until you understand them then pastors remark luck favors prepared mind will favor you.

When world passes you by you are not happy. Richard saw at Bell Labs they were getting from relays to electronics and analog people weren't happy because of their unwillingness to convert.

If you do that which society wants and needs, society frequently recognizes it and repays you somewhat.

You don't always do what you want to do, you do what you believe needs to be done and needs badly

1. If you want time invarient systems then eigen functions or translations are sins or cosines or complex explonentials.
2. If you have linear dynamic system, eigen functions are complex exponentials.
3. If I have a equal space data and I have a pure sinusoid of high frequency it will be aliased into a pure frequency of low frequency.
4. Transfer function is nothing but eigen functions.(Nyquist theorom)
5. For any set of orthogonal functions the best least square fit is the fourier series.

You know least squares but you don't.

You are looking at the data through a window. You never see nature itself. It's always through a window usually the instrument but sometimes instrument is your head your inability to think. You impose things on what you see because you think that way and therefore that is what you see. You never get to see reality.

He is talking a lot of math going over my head.

At what order of magnitude you will say it's new? Hamming says it's 10. You walk at 4 km/hr, automobiles at 40 and Aeroplanes at 400. There were people who said well computing isn't new it's just little bit faster. Those people never made contribution to the field. People who made contribution thought it was new.

[!tip] Musing Try to recognize the rate of change in anything.

I became famous to a great extent because when I went to Bell Labs in so far as I could I would work with the interesting smart people and I would avoid the dummies.

[!important] Proposition What you learn from others as you learn from me past now you can use to follow that which you learn for yourself you can use to lead

Feynman think it through physics for himself. He didn't believe anyone. Hamming did that same for calculus.

[!note] What you learn for yourself you can use it to be a leader. What you learn from others you'll use to be a follower.

How do I get you to digest things for yourself so that you will become a leader and it is just that little extra what you know you will home and think about you make it your own by re-digesting the thing until it is yours by your ways of thinking that is available for you use if it's just by memory you'll just remember what I said and that isn't good enough. so you got the message very very simple it's those two things luck favors prepared mind or the other thing learning the fundamentals going a little bit further not taking what said but thinking for yourself and that is the difference between first class leaders and sacrifice people.

People have to show off how much they know. You will find in many talks receive the speaker is showing off how much he knows and is not trying to really communicate with you.

Cooley give simple Idea to hamming to do on computer he had to use cards then and thought it can't be done and is a bad idea. Then later when new computer arrived and Cooley mentioned again hamming to do it. He didn't do it as he thought it was a bad idea and can't be done. He forgot that there has been progress in computers and with internal programming it might be easier.

[!note] When you decide something is impossible and years later somebody says you don't clearly say it's impossible. Go back through the reasons are they still true.

Fourier Analysis implies linearity of the underlying mechanism.

[!note] When a field of knowledge is extended to a new area the new definitions are often inappropriate.

Everyone wants to fit everything into old. They don't want to think new thoughts. Is there something really new?

The taste of recognizing when something is fundamentally new and when it is something old with a new flavor a new aluminum sheeting instead of steel or something like that is a very hard thing for you to do but that is a fundamental problem you will have to face in your career again and again and again is this a new thing or is the same old stuff with a new coat of varnish put it again with a quick if you think it's nothing new you're not likely to contribute if you think it is to be new you're likely to spend most your time chasing something it's not there but it's the only way to be the leader.

[!tip] Find people doing important things and help them get the important results.

Most people are not doing any important and never going to do anything important so it doesn't matter whether you help them or not.

We are specializing more and more and more as a result people know less and less about general business. How to see the bigger picture and keep the modest in the bigger picture?

You have to start exhibiting those characteristics of leadership and trustworthiness and open mindedness and other things I'm trying to inculcate you.

[!success] You need to know a great deal about everything.

And you have to ask yourself how does this apply to that or where else does it apply?

Nobody has told me this but I'm telling this to you. Start fighting compartmentalization(Specializations in 2023) and limited knowledge and start thinking of how generally applicable it is.

After all thing digital filter lectures I'm understood one thing and that is math is highly interesting and essential.

Simulations

It about What If..., until now we have been doing it mentally. What is we do this, or what if that happened. Now we do computing ones. It is one of the main use of computers.

It is faster, cheaper and more importantly they can do what you can not do in lab. They are more accurate, often better. Simulation can answer things you can not even hope to measure or things which experiments you can not do, I can do on a machine.

In a field of simulation if you don't have experts or expertise to come up with equations you won't get reliable answers.

It is easy to do simulation for definite problems like designing atomic bomb but quite difficult in situations like weather prediction as they are very local and if butterfly flaps it's wings in japan it will change the location of storm in US.

Frequently fewer answers understood are much better. Galileo Complained back in the early 16 hundreds that instead of looking at nature they read what Aristotle said I should do and they believed whatever Aristotle said. And he argued that the way to get things going is to look at nature herself and see.

We are in danger by using these simulations and using these formulas I get from others. We must look at the world periodically.

Start with simple simulation and then grow and get detailed. An active aggressive mind can add a great deal to simulation. Somebody who just does what they are told adds nothing. I can simulation calculate the ideal situation and that often shares more insight and understanding even if its not relevant to reality then reality will.

Jargon is a device instinctively used to exclude the outsider. When you plan on doing simulation in an area go out and master the jargon. Jargon is essential. Jargon can confuse yourself, beware of it.

There is no substitute for somebody who understands the problem.

Is the original problem posed right? Did he give me something wrong? Have I got the right answers? and does he understand the results?

Older people don't get new ideas well. You much know the laws of what you are simulating or else you aren't going to get there very well.

Reliability of simulators are vital. With the power of decision rests the responsibility.

Analogy and digital neural nets are pretty much equivalent. You will find in time that analog computing is good.

You can basically design with inaccurate components and get accurate results.

Just because problem looks like it can't be done. Don't give up.

You have not been trained to replace theory with no theory. You have not been trained at all to recognize no signal is just plain noise.

You might be finding what you want to find. It's very very difficult not to find what you want to find. You have preconceived notions and you see what you expect to see you do not see what you don't want to see.

How do you recognize there is an opportunity for something and when do you not look? You are born into an age of change. Accepted theories one day are not appropriate the next day.

Alex from psychology a friend of hamming did experiment where he had 12 switches and green and red light. Source of light was random but 20 people one by one had to give theory and had to build on top of previous ones and he found that everyone found meaning although there was no theory it was random. Now you do this yourself in your life, often you find meaning in events which are probably meaningless. It's quite hard to be objective. You must doubt yourself a great deal and search positively am I finding what I want? be very careful.

Fiber Optics

New areas will arise but you won't have time or energy to become an expert but you cannot afford not to know.

If you are at place like MIT or Ball Labs always a lot of interesting talks would be happening. But if you do that you will always end up learning but not doing anything. But before I went for this optical fiber talk at Bell Labs I knew few things about it like Graham bell sent voice over light. He knew about internal reflection. He went cause he knew its bound to be important to telephone company and he was bound to get problems on the subject.

Whatever you have learned in electrical engineering about frequency and it's analysis isn't appropriate for future communications Soil tons are not frequencies they are different device entirely.

Computing was changed completely when we got much more memory if you get the price of switching download has been the high part for long long time. If you bring that down to be very cheap how will you design the computer?

There must be thousands of people who rally around the idea and accept it and act on it what the leader is the guy who has the idea articulated carefully and notice the second part history of science shows that a lot of people had ideas they did not get them across they rotted and had to be rediscovered I told you the fast Fourier transform if I didn't I should have it went back to Gauss you see LMS lobbies they didn't exploit it when twokey really got after it when the time was right he did it but now it's a wide widespread use well my problem is exactly this you will see a lot more than I need of new innovations the pace of technology is indeed increasing I had trouble keeping up you're going to have more trouble but if you will not keep up two things are true you'll be left behind and you will not be one of the leaders you won't be one of the people who matter and my task is to make you people who will matter.

You are going to have to learn a large number of things or you're going to be playing obsolete.

CAI Computer Aided Instructions

Can computers help with teaching? Can computers help to learn quicker and better way?

Hawthorne Effect.

There are two aspects of education and you have to know about them

1. That what you learn from others you follow and that you learn for yourself you can use to lead.
2. The ability to learn rapidly was an important part of education.

Mathematics

It's worth while to stop and ask yourself What is that you are doing? What is the subject you are studying? What is Mathematics? What is Bell Labs?

Mathematics is different for different people. Many things cannot be defined sharply. mathematics is the language of clear thinking. We invented this artificial language quite different than any natural language.

Math is essentially an Universal language. Mathematics is in the realm of ideas which is not real at all. Math is continuously changing.

When you start rigor you have given up all meaning there isn't any truth in mathematics not one iota.(Mathematics a loss of certainity)

Five schools of math

• Plato(Math ideas and theorems all exist in space of ideas where its waiting to be discovered. He says Idea of chair is more real than chair itself as it can not be destroyed )
• As formalism(Hilbert says when rigor enters meaning departs. There is no meaning to anything. Its all manipulation of strings and symbols)
• Logicians(Russell, pure math)
• Intuitionists (There maybe a ground between yes and no. If something isn't a it isn't a. If we don't have any intuitive feeling about math. How am I to prove it?)
• Constructvist

None of the five theories are correct and none of them are wildly accepted.

Despite the fact that there no truth in mathematics somehow we successfully predict atomic bombs, guided missiles and everything else and we did pretty well.

Gödel said that it's impossible for you to prove within the system that that system is self-consistant.

Non-euclidean space.

Everything really worth knowing can not be said those things which are really valuable.

Quantum Mechanics

Some physicist believe that we pretty much know all basics of physics there to know. But they would had to agree on fact that we don't know anything about 90 to 95% of matter in universe.

So physics doesn't know everything yet. Question is how much do they really know?

Science never tells you why, it tells you what.

Max plank, Schroedinger, Heisenberg, Eckart carl.

You can have different theories which account for the same phenomena. Given phenomena, its not possible to go to a unique theory.

You look at the world, how it behaves. You say, "Oh, that's how great work is done. Let me behave that way. Maybe I will be involved in some other great work."

You cannot go from data to unique theory. Much as you might want to , it isn't possible.

How can light be both particle and wave?

Copenhagen Interpretation of Quantum Machenics. What you mean by probability?

Even if you cannot or our minds are not wired to understand some phenomena. That does not mean that we can't build mathematical structure which will enable us to predict reliably.

That's roughly what we have done in quantum mechanics.

Emerald Buddha.

You religion will to some extent, determine what you will believe about the world.

If you think that there is two different things, how does this materialistic world changes because of this other world? Where does the interaction occur? how does it happen? Does it affect cats and dogs?

Alain Aspect

**Two particles, once connected and tangled, are permanently entangled in one behavior can instantaneously transport the other. **

Pedolsky and Rosen -> EPR paper Bell inequalities paper

St. Augustine 1680

Blind programmer MIT

So we are beginning to be able to hold single atom right there. And know we've got one atom right there. Up until very recently we couldn't.

Every linear theory must have an uncertainty principle.

Creativity

Now creativity, novelty, originality are words our society tends to value.

Before I go further, I should tell you that primitive tribes don't value those words. You are to follow what elders say and what they always did do. And is no different in good many large organizations.

One of the goal of this lecture is to increase the chance of you being a creative person.

There is a difference when you multiply 10-digit random numbers and find the biggest prime number. Later would be called new as it's difficult to do. There is no novelty in multiplying large numbers although that's something one has never done before.

I can paint a picture and it will be novel and new but won't be great art.

There was a time when very few people understood relativity. But it got bigger appreciation later on. And same seems to be true for Art at the time people laughed and they were poor but now their art is worth millions.

We all have our failings in recognizing new things very badly. How difficult is it to associate two things together?

Creativity involves "Psychological distance": How difficult is to associate 2 things that are distant from each other?

That your creativity is in fact bringing things which are psychologically far apart and bring them together and say look they are related. This is all we are really doing.

There are things which you can talk about read but you have to experience and I think creativity is one of those things.

Saturate your subconscious with the problem, by thinking about it all the time. (luck favor the prepared mind).

Those who care are more likely to do something than those who don't care greatly. Emotional involvement is very necessary and keep your mind in saturation with thinking about the problem.

When I'm stuck I do this, ask yourself what would an answer look like if I had it. Have I got all the information? What does it really depend upon?

After you got it. Clean up your idea remove the nonsense and communicate it well for other people to understand.

Now probably most useful thing in creativity is analogy. Analogies can help you even if they are loose analogies. They work as suggestions.

Learning things in the framework you learned them could make you overlook things. In the act of learning, Try and reconstruct your thinking in different ways so you remember it better. Put different hooks on the idea. Don't be trapped in that framework that gave you that knowledge in the first place.

I've always search wherever it was to find those people who would simulate me to think better. Who are the people to whom I told something they would say something back instead of "yeah, That's interesting!" doesn't help you a bit.

You pick your friends so they will simulate you to think.

You have to take charge of yourself.

If you can't drop a problem and if you get first bad problem that's the end. And classic example is our boy Al Einstein. He had a lot of good ideas, he produced a lot of things but once he hits unified field theory, the second half of his life went down the drain and that problem he didn't get anything. It was the wrong problem at the wrong time in the wrong way.

I have often argued that Oppenheimer at the institute should have called Einstein to his office. A little boy do me a favor drop that unified field for six months a year and work on anything else at all but just drop that. Would he be able to do it I'm inclined to think he would have but who thinks they got enough nerve to tell Einstein how to run his own business. Oppenheimer might have but he didn't as far as I know.

On the other hand if you drop problem too soon somebody else comes by and does it.

The difference between strong-willed and stubborn is very thin. But it's essential that you know when to drop a problem.

There is an example of Institute for Advanced Study. Where you are given free time, lot of money and everything you would imagine possible. But it killed more good people then anyplace every created.

Ideal work condition of what you think are ideal are not. Often unpleasant work conditions are the one that stimulates you. So don't give me an excuse.

You have to know yourself to manage yourself. You have to take responsibility for yourself. You have to understand what you can and cannot do.

Most great scientists, their best work was done surprisingly young. In Some field maturity is the best thing but in AstroPhysics, Mathematics and theoretical physics where raw creativity counts youth is the great advantage and experience is not.

Experts

An Expert is one who knows everything about nothing whereas generalist knows nothing about everything.

Expert almost wins against the generalist almost always by the following devices

1. You use lot of jargon which other doesn't know
2. You invoke basic principles in your field which may be totally irrelevant that sounds good

Khun -> Scientific Revolution

Cosmology is very interesting science. You got only one sample and you can't experiment.

There were lot of book written on relativity which were against relativity. Once Plank said, We didn't convert them we outlived them. New Ideas are greatly resisted.

If expert tells you something can be done. It is probable it can be done. But If expert says it can't be done. It's better to pay and get an expert who may tell you it can be done.

My experience at Bell Labs is that experts have been marvelously wrong lots of times. They don't understand the problem. They force the situation into a situation they know and then look at it that way from their trained eyes.

Most of the time bright new idea come from out side the field not within. Take archaeology and carbon dating? Carbon dating was proposed by physicist and was opposed to great extent by archaeologist.

Again and again next step forward is done by somebody not in the field.

What is your strategy? If you listen to all of them you will get nothing done. If you ignore them you want be a part of next big thing. There is no easy answer. If you very much want to be connected with the next big step forward then you listen to more kooks.

I'm not interested in competition I'm interested in what we can do together. I try not to get in the way of next generation.

History says that people who created the idea seldom understands it.

Life is full of change keeping is one of the major problems and it troubles the expert when he becomes the expert he freezes the knowledge at that state and does not go on.

Keeping an open mind everything is possible is not satisfactory you have to put some probabilities to it and those are hard. But if you don't put probabilities oh that's possible. you won't get very far.

Unreliable Data

It has been my experience and of many others that data you get is not as reliable as it's advertised. Now this is serious matter as not only in simulations that you suppose you have reliable data but many other decisions which you want to make the data is not accurate.

Why should anyone believe that test equipment that you are using is as reliable as what you are testing.

accelerated raise temperature for life testing. In chemistry every 17 degree Celsius rate of reaction doubles.

How will you decided that it's going to last as long as its supposed to do.

Chain is as strong as weakest link

In any long-term time series what is being measured if often changing

Poverty is still there and definition keeps changing and it will be there.

Morgenstern

I'll say again, It's better to take a small sample accurately done than a big sample poorly done.

If you think economic data is bad consider social data it is worse.

Guns are easy to count but attitudes are harder.

- The gap between science and engineering has shrunk.
    - There is often no time to field-test new equipment. We can choose either to work with uncertainty and use new equipment anyway, or to become obsolete.
    - Reliability testing: is the testing apparatus so much more reliable than the samples being tested, that we're confident that the results are about the reliability of our tested samples, not about the reliability of the testing apparatus?

- Statistics:
   - Measurements are less accurate than they say: "90% of the time the next independent measurement will fall outside the previous 90% confidence limits."
        - Calibration of equipment, other than for well-established measurements (eg. calibrating a length of 1 m by some number of wavelengths of a certain frequency of light), often relies on _fine tuning for reproducibility (low variance)_. And then the published confidence intervals are based on this variance, which had been subject to Goodhart's law. "When a measure becomes a target, it ceases to be a good measure." Targeting low variance means the final variance is not a good measure of the true uncertainty.
        - Economic data are similarly collected for a purpose, eg. a country receiving foreign aid may optimise their data in order to receive more foreign aid.
    - Measurements often become obsolete as the circumstances change. We can either change the definition (and future measurements become incomparable with past measurements), or let the measurement become irrelevant.
        - eg. index rebalancing, consumption basket changes lag behind reality
        - eg. modern wheat and 1600s wheat are both measured in bushels, but are rather different products
    - Small samples, carefully collected, is often better than large samples, poorly collected.
        - But organisations often execute a pilot / trial differently from how they would execute a full-scale job / study. That could give different results.

- Sometimes, the mere presence of a high-ranking / highly prestigious person can affect the very thing you want to measure, eg. morale.

Systems Engineering

A man was looking at the cathedral being built. He went and asked one by one everyone on what they where doing, sculpture said he was sculpting and so on. But at the end when he came to a little old lady who was sweeping the floor and he asked her what are you doing. She said I'm helping build a cathedral

Same goes when you ask prof at university on what they are doing they will rarely say, he is trying to education young for their future.

Now you may say in both parables that of course bigger goal was always understood, but I doubt you really believe it.

We often forget the bigger picture.

Well systems engineering is an attempt to look at the whole as a whole at all times. That's what it is.

There are lot of people that will tell you how to play tennis from start to end but they can't play it that way. Same is for systems engineering, there are very few good systems engineers but large amount of people who can say all the words.

So I claim to you the problem is very very difficult. You know what you want to do perfectly but you can't do it.

Now the first rule of systems engineering is one which you won't believe. If you optimize a component you probably ruin the system's behavior.

Economy of scale

Part of systems engineering job is to prepare for changes so when they occur you can make them gracefully. We have to think carefully not over design one part and under design one part.

Now I got another rule, closer you hit design conditions the worse in overload. Graceful degradation of system under overload is essential. Bob Westerman has written some notes on systems engineering.

One of the things about systems engineering is that the people with various skills must be given free time now and then to go back and sharp up your skills in their various technical areas.

You learn systems engineering by osmosis by doing.

One of the things you need to do to be a systems engineer is be some successive to human side of engineering.

Hamming's 3 rules of systems engineering (paraphrased):
    1. optimizing individual components will probably hurt the system as a whole
    2. prepare for changes — flexible, modular designs
    3. build in buffers for graceful degradation when overloaded — exactly meeting the specifications makes the system less robust.

- Systems and solutions change each other — a continual evolution.
    - Presence of a solution changes the environment and produces new problems.
    - A solution to the original problem usually produces both deeper insight and dissatisfactions in the engineers.
- Systems engineers need to block most of the local optimizations of the individuals and reach for the global optimization for the system. But these blocks change the environment, and the individuals change their local optimizations! (see previous point)
    - Human psychology to better align individual humans' incentives with the system's goals.
    - Motivating people to learn something new: figure out how to appeal / make it useful to them.
        - How is their morale? What do they consider important?
        - Does the new thing only work under ideal conditions? How well does it work in the field?

- Systems engineering needs domain specialists, who, between jobs, need to return to their specialties to maintain their expertise. But they face pressure from management to stay "just a little longer" to put out yet another fire.
- Systems engineering is difficult to teach; it must be lived. New people need to be brought into systems engineering teams (and old people gradually removed).

- The client knows the symptoms but probably doesn't know the phenomenon or the root causes. The systems engineer needs to theorize, formulate / define / identify the phenomenon (like a doctor synthesising information to come up with a diagnosis).
    - Without good theorizing, it's easy to solve the wrong problem.
        - Better a rough solution to almost the right problem, than a precise solution to a wrong problem.

- Example of emerging phenomena (concepts and objectives) with Nike missiles:
    - 1 plane, 1 missile:
        - objective: hit the plane.
    - A fleet of planes, a battery of missiles:
        - objective: hit as many planes as we can
        - new concept: coordinate missiles to hit different planes.
    - A nation to defend, many missile batteries:
        - new concept: how much would it cost the enemy to cause 1 value-unit of damage?
        - objective: make every value-unit of damage equally costly to the enemy.
            - Concentrate defense on valuable / vulnerable targets
            - When every value-unit of damage is equally costly, the enemy cannot gain an advantage by cleverly choosing targets.

You get what you measure

It means the kind of measurement you will make will affect the organization.

What happens when you put measuring system?

Organizations need outsiders in board. Why not apply then? -> hamming didn't told, I think I should.

How to find a natural scale? statistician won't do you any good.

You can add two herds of animal with arithmetic scale but that shouldn't be used for judging other things. And I suggest you that different scales are needed for different things.

• We are likely to end up measuring something that can be measured easily, accurately, and reproducibly, instead of something relevant.

• Before setting up a measurement, think about long-term effects under Goodhart's law. What are individuals (components of the larger system) incentivised to do?

• What calibration curve we choose determines what distribution we get.

  • eg. "IQ is normally distributed with mean 100 and standard deviation 15" because that's how it was calibrated.
  • eg. an exam on a single narrow topic with all questions equally hard is likely to get approximately a bimodal {0%, 100%} distribution of scores - candidates either know the topic (100%) or don't (0%).
  • eg. an exam with a uniform spread of question difficulty is likely to get approximately a uniform distribution of scores.
    • Information theory: uniform use of all available symbols / the full dynamic range maximises the amount of information.

• Human judgements vs machine measurements:

  • Human judgements are reasonably good and reproducible, and can informatively collapse a vast multi-dimensional situation to a single numeric scale (eg. asking Alice and Bob to rank all candidates for their suitability at being the next leader), but asking for explanations is bad because people have difficult-to-verbalise intuitive judgements.
  • Machine measurements are consistent, which can allow other smaller details to come to light without being overwhelmed by noise in human judgements.

• Personnel:

  • Personnel pipeline: the people who are attracted to the outward appearance of the field / company apply, they start and then rise towards the top.
    • eg. Math has the outward appearance of detail-oriented minutiae (high school algebra / differential equations), attracts that kind of people, but these are the only people available even later when it switches towards big-picture / general unification of concepts / creativity in pure math / research
  • Researchers with original ideas in eg. physics is likely to also have original ideas in fashion / hairstyle. If we judge them by conforming to unofficial dress codes, we would get conformists instead of original thinkers.
  • People tend to hire / promote people like themselves. Be aware this tends to entrench a particular school / style of thought.
    • A non-homogeneous team can make people more conscious of their choices, which might be a good thing.
    • In a highly homogeneous team, outsiders' recommendations may be cited if the insiders agree, and ignored if the insiders disagree. Hint: what happened to the past 10 - 15 years' external review recommendations?

• Organisations

  • Random inspections? The air force base has a radar, you can't surprise them. But giving only hours of preparation time gives a better reflection of true readiness than giving days of preparation time.
    • Using results of inspections as input in business simulations / war games could be misleading. We'd train the leader to win under ideal circumstances, not under realistic circumstances.
  • AT&T, while being a monopoly, used internal surveys of operating companies to create internal competition
  • Organisations might have stupid "habits" which are resistant to change, eg. AT&T cannibalizes parts from nearly-complete builds, puts them in end-of-quarter shipments, to game quarterly shipment numbers.
    • Go out and look at reality (eg. the physical warehouse; speak with rank-and-file employees) to verify what is actually happening.

Before you find yourself putting in a rating system ask yourself what will be the long term global effects.

How do we know what we know

if indeed we know it

It's very old topic and has been discussed for thousands of years and in philosophy it's given name Epistemology -> Science of Knowledge

There is a rule in biology, ontogeny recapitulates phylogeny meaning the growth of the individual from fertilized egg up to you repeats in a very schematic way the evaluation behind you.

Now you can get awful far without understanding what you are doing.

I'm trying to convince you that there is no certain knowledge.

Wishful thinking is one of the biggest curse man has to put up with.

You and your life

Once not famous what you do will be taken away from you. So it's necessary to do something outstanding.

Luck favors prepared mind.

Edison said genius is 99% Perspiration and 1% Inspiration. I would to great extent say that it is constant hard work that does it.

I had resolved to work with important people.

Now the most important thing about great people is that they believe that they can do great work they have confidence in themselves.

If you don't think you do great work it's not likely that you're ever going to do it.

If what you are doing is not important not likely important why are you doing it?

You must work on important problems.

What appears to be a defect no so. If you can turn the problem around you can turn to great success.

Race is not to the swiftest, the guy who works hardest doesn't win, the person who works on right problem at the right time in the right way is what counts and nothing else.

When problem is ripe? What problem is ripe and how to go about it.

I advise you to stop and think what are the important problem? what is going on? What the nature of what you are doing? What are the fundamental behind.

Most scientists have 10 to 20 problems in mind.

The way you learn as far as I'm concerned is every time you go to a talk you listen not only to the talk but to the style that's done. What talks are effective? Why were they effective? What aspects of speaker can you adapt?

Change does not mean progress but progress requires change.

[!summary] what this lecture is all about revivals preacher preaching well now I've told you things how to succeed no one ever told me these things I've been telling you nobody I had to find it for myself I've told you how to succeed you have no excuse for not doing better than I did.

links to check out

• [ ] https://mihirchronicles.com/the-art-of-doing-science-and-engineering/
• [ ] https://www.notion.so/blog/a-roundtable-on-richard-hamming
• [ ] https://en.wikipedia.org/wiki/Systems_theory
• [ ] Finding most influential people in every field.

https://news.ycombinator.com/item?id=34591291

Books by Thomas s Kuhn -> Intellectual Path, The structure of Scientific revolutions