My dad (an engineer not a mathematician) would use Newton-Raphson[1] to solve basically any problem that wasn’t very obviously linear. When I was a kid, some of my first programming memories were my dad getting me and my brother to implement Newton-Raphson in basic on an HP85a, getting me to implement Newton-Raphson in rpn on an HP calculator, debugging my dad’s (genuinely revolting) basic program[2] which wouldn’t run but (who would have guessed?) used Newton-Raphson to compute something or other.<p>He basically learned the one numerical root-finder and how to evaluate basic second derivatives and he was set for life on all the problems a career in chemical and process engineering could throw at someone.<p>[1] <a href="https://sheffield.ac.uk/media/31988/download?attachment" rel="nofollow">https://sheffield.ac.uk/media/31988/download?attachment</a><p>[2] He learned to program in FORTRAN and lived by the maxim that a determined FORTRAN programmer can write FORTRAN in any language.
My dad was also an Engineer. He was also a Fortran fan.<p>Once he asked me to explain OOP. After I explained the basics he said it was useless and never looked back.
Get your dad a copy of the classic <i>Scientific and Engineering C++: An Introduction With Advanced Techniques and Examples by John Barton and Lee Nackman</i>. It was written to introduce C++ to Fortran programmers using examples from scientific/mathematical domains. The fact that it is old (from 1994) makes it better suited for folks from Fortran (or other languages) since there is none of the complexity of "Modern C++" to confuse them. Check reviews on Amazon etc.
Chad Dad
I guess as a typical engineer (not CS person type of engineer or software engineer) it is easy to think that. One might be working with machines or buildings and so on, all which require _calculation_ of processes. Those are typical cases for "just write a correct function", possibly one that takes many things into account. For such scenario OOP is truly useless and only over-complicates the matter. However, when we get to simulations, where maybe there is no known formula, or the precise calculation would be too expensive, then OOP can make sense. Doesn't have to, but could.
Relatedly, I've found Newton-Raphson is a great example of an algorithm where Knuth's "I have only proven it correct, not tried it" rears its head very prominently. The obvious implementations can work flawlessly on toy examples and then fail miserably on real-world examples.
Nobody has enough memory or patience for third order derivatives so Newton's method (aka Newton Raphson) it is.
The modern world of autodiff actually makes real second and third order derivatives fairly cheap to compute.<p>The stuff that is actually used most commonly, only uses first order derivatives though (gradient descent, Levenberg-Marquardt, Kalman filters...)
I think this is true for engineers as well! I enjoy getting to know the "theme" of my favorite coworkers over the years. There was:<p>* The fellow who always looked for the simplest hack possible. Give him the most annoying problem, he'd pause, go Wait a minute! and redefine it to have a very easy solution. He typed very slowly, but it didn't really matter.<p>* The one who truly loved code itself. He would climb mountains to find the most elegant, idiomatic way to express any program. Used the very best practices for testing, libraries, all that. He typed very fast.<p>* The former physicist who spent all his time reading obscure mailing lists on his favorite topics. His level of understanding of problems in his domains of interest was incredible.<p>I could go on and on! It's such a fun taxonomy to collect. All of these friends were marvelous at solving their particular flavor of problem.<p>As for myself, I like to think that my "trick" is to spend a long time poking at the structure of a problem. Eventually the solution I was looking for doesn't matter anymore, but the tools I developed along the way are pretty useful to everyone!
Here are a few more.<p>* The (brilliant) infrastructure engineer who described his modus operandi as 'I read stuff on Reddit and then try it out.' This engineer is now worth, as a conservative estimate, in the neighborhood of $50 million. So maybe more of us should be doing that.<p>* Another infrastructure engineer, also very effective, who made a habit of booking an external training session (sometimes a series, weekly) for how to set up and integrate every piece of technology we used.<p>* An engineer (this one is quite famous, and we have only interacted professionally a few times) who simply wrote the best comments in the world. Every method was adorned with a miniature essay exploring the problem to be solved, the tradeoffs, the performance, the bits left undone. I think about those comments quite often.<p>As an addendum, though, I will say that the best engineers overall all shared a trait - they kept trying things until they got something working. That alone will take you pretty far.
For me, it's tracing code/pipelines to figure out how a result was produced, typically in the context of that result being wrong somehow. Go To Definition is the most useful function in any editor.<p>I'm always surprised by how frequently colleagues don't think to do this and are left helpless.
This reminds me of my further theory that everyone needs one 'heavy' and one 'light' technique. The 'light' technique is something that often works well as a heuristic and can be an effective unit of iteration. The 'heavy' technique is something that you can fall back on in difficult cases, something that can reliably solve hard problems, even if it's slow.<p>Sometimes the heavy technique is: just ask someone else. ;)
> <i>Sometimes the heavy technique is: just ask someone else. ;)</i><p>For a lot of people I know, this is the <i>light</i> technique!
You jest, but that's how my sister gets through life, and it's always fascinated me.<p>She's incredibly intelligent, but more importantly she's a <i>phenomenal</i> social networker. She always has someone to call to ask about any question or solve any problem that comes up in life, and she's great at connecting these people with each other when they have problems of their own - so they all <i>want</i> to help her with whatever she needs, just to gain access to a pool of people they themselves can talk to.<p>What do you do with a skillset like that? I honestly don't know - something in leadership, probably, something where finding the right people and setting them to work is the most important skill of the job.
That wasn't in jest. I worked in a place where this was a norm. Nothing was properly documented, instead everyone would just ask and answer questions on chats; somehow, this actually kept velocity high.<p>Found it really hard to adjust to that. I'm the kind of person that prefers to research things on my own, find hard references and understand context. But there, this was the wrong approach.
For me the heavy technique is integer linear programming.<p>I’m not a software developer anymore.
Another example: debuggers.<p>It's amazing that a lot of new developers don't know how to use them at all! I'm not even talking about using the command line gdb, but just the basic "Set Breakpoint" feature and attaching to processes.
[dead]
Feynman wrote in his autobiography that much of his success came from having different mathematical tricks than most of his peers. So when they were stuck, he could sometimes make progress.
Interestingly, one of his famous tricks of computing complex integrals by parameterizing[1] and then differentiating under the integral is known as “Feynman’s trick” in his honour spite of it having been invented by Euler over 250 years before.<p>[1] <a href="https://zackyzz.github.io/feynman.html" rel="nofollow">https://zackyzz.github.io/feynman.html</a>
He had many tricks in limited area. All his his tricks were limited to classical calculus and nineteenth-century mathematics. He didn't do anything fancy.<p>His style was always the same, he just mastered it really well.
Feynman used to read his own books. When asked he said, "it's all in here". He used to revise and refresh his own understanding.
My impression from math lectures as a CS student was, that you have to know the tricks, otherwise you just sit there dumbfounded, not knowing how you can make progress. Obviously, there will also be some kind of mathematical understanding of things at play, but a lot of the math stuff is pattern recognition and applying tricks. If you don't know them, it becomes very frustrating. I've not ever seen a way of teaching math though, that focuses on teaching learners these tricks or techniques. They always fell out of things on the side or the teaching person assumed you know it, or was surprised, that the learning people were not able to see it, or able to apply some trick. Or they just didn't care, as long as a small portion of the learners somehow were able to solve it, because then it obviously cannot possibly have been their way of teaching, but other students must have been lazy...
I remember annoying one of our professors at the university. Whenever he was discussing a problem and found me falling asleep, he called my name. I woke up, said Chinese remainder theorem and with like 90% success rate it solved the problem handily. Yes, it was an Algebra class. We were still surprised how well it worked.
One of the answers links <a href="https://www.tricki.org/" rel="nofollow">https://www.tricki.org/</a>, which describes itself as a 'Wiki-style site with a large store of useful mathematical problem-solving techniques.' no longer maintained, but looks neat.
One surprise of reading this thread was learning that Terry Tao is an active user of Math Overflow!
My lecturer was once unable to solve a problem at the blackboard.
After trying for some time, he said he needed a short break to go back to his office to look at his notes. He brought the notes, and there he found a hint he had written for himself: "Use a trick"
For programmers I would say graphs or at least thinking in graphs is a common one.<p>Some would say SAT is also a handy trick but I’ve personally never used it.
There is a joke in applied mathematics that we’re like Taco Bell. We all use the same six ingredients, mixing them in different ways.<p>For myself, I’ve found several techniques I use over and over again. Some of this is a “when you’re a hammer, everything looks like a nail.” But fundamentally there are only a handful of ideas. One professor of mine once said the only groundbreaking result in the past few decades was compressive sensing.
For most of my computer science PhD the “trick” was just to get the inductive definition to work, and then how to tweak it for the next paper. Or, get enough structuret we can do an “abstract nonsense” proof[0].<p>[0]:<a href="https://ncatlab.org/nlab/show/category+theory#AbstractNonsense" rel="nofollow">https://ncatlab.org/nlab/show/category+theory#AbstractNonsen...</a>
From a comment:<p>> These methods can be combined. First generalize the problem, making it more complicated. Then simplify along a different axis. –
Stig Hemmer<p>Very relevant to software design too.
I like the trick of adding zero to an equation.
A mathematician tricks.always.
There is no point to untrick the trickster. Misuse of your time
[dead]
[dead]