Steve Brunton
Steve Brunton
  • 486
  • 21 635 345
Residual Networks (ResNet) [Physics Informed Machine Learning]
This video discusses Residual Networks, one of the most popular machine learning architectures that has enabled considerably deeper neural networks through jump/skip connections. This architecture mimics many of the aspects of a numerical integrator.
This video was produced at the University of Washington, and we acknowledge funding support from the Boeing Company
%%% CHAPTERS %%%
00:00 Intro
01:09 Concept: Modeling the Residual
03:26 Building Blocks
05:59 Motivation: Deep Network Signal Loss
07:43 Extending to Classification
09:00 Extending to DiffEqs
10:16 Impact of CVPR and Resnet
12:17 Resnets and Euler Integrators
13:34 Neural ODEs and Improved Integrators
16:07 Outro
Переглядів: 29 138

Відео

Neural ODEs (NODEs) [Physics Informed Machine Learning]
Переглядів 46 тис.14 днів тому
This video describes Neural ODEs, a powerful machine learning approach to learn ODEs from data. This video was produced at the University of Washington, and we acknowledge funding support from the Boeing Company %%% CHAPTERS %%% 00:00 Intro 02:09 Background: ResNet 05:05 From ResNet to ODE 07:59 ODE Essential Insight/ Why ODE outperforms ResNet // 09:05 ODE Essential Insight Rephrase 1 // 09:54...
Physics Informed Neural Networks (PINNs) [Physics Informed Machine Learning]
Переглядів 40 тис.21 день тому
This video introduces PINNs, or Physics Informed Neural Networks. PINNs are a simple modification of a neural network that adds a PDE in the loss function to promote solutions that satisfy known physics. For example, if we wish to model a fluid flow field and we know it is incompressible, we can add the divergence of the field in the loss function to drive it towards zero. This approach relies ...
SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning
Переглядів 14 тис.Місяць тому
SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning by Nicholas Zolman, Urban Fasel, J. Nathan Kutz, Steven L. Brunton arxiv paper: arxiv.org/abs/2403.09110 github code: github.com/nzolman/sindy-rl Deep reinforcement learning (DRL) has shown significant promise for uncovering sophisticated control policies that interact in environments with complicated dynamics, such as sta...
AI/ML+Physics: Preview of Upcoming Modules and Bootcamps [Physics Informed Machine Learning]
Переглядів 16 тис.Місяць тому
This video provides a brief preview of the upcoming modules and bootcamps in this series on Physics Informed Machine Learning. Topics include: (1) Parsimonious modeling and SINDy; (2) Physics informed neural networks (PINNs); (3) Operator methods, like DeepONets and Fourier Neural Operators; (4) Symmetries in physics and machine learning; (5) Digital Twin technology; and (6) Case studies in eng...
AI/ML+Physics: Recap and Summary [Physics Informed Machine Learning]
Переглядів 14 тис.Місяць тому
This video provides a brief recap of this introductory series on Physics Informed Machine Learning. We revisit the five stages of machine learning, and how physics may be incorporated into these stages. We also discuss architectures, symmetries, the digital twin, applications in engineering, and the importance of dynamical systems and controls benchmarks. This video was produced at the Universi...
Using sparse trajectory data to find Lagrangian Coherent Structures (LCS) in fluid flows
Переглядів 8 тис.2 місяці тому
Video by Tanner Harms, based on "Lagrangian Gradient Regression for the Detection of Coherent Structures from Sparse Trajectory Data" by Tanner D. Harms, Steven L. Brunton, Beverley J. McKeon arxiv.org/abs/2310.10994 The method of Lagrangian Coherent Structures (LCS) uses particle trajectories in fluid flows to identify coherent structures that govern the behavior of the flow. The typical metho...
AI/ML+Physics Part 5: Employing an Optimization Algorithm [Physics Informed Machine Learning]
Переглядів 14 тис.2 місяці тому
This video discusses the fifth stage of the machine learning process: (5) selecting and implementing an optimization algorithm to train the model. There are opportunities to incorporate physics into this stage of the process, such as using constrained optimization to force a model onto a susbpace or submanifold characterized by a symmetry or other physical constraint. This video was produced at...
The Future of Model Based Engineering: Collimator 2.0
Переглядів 18 тис.2 місяці тому
Learn more at www.collimator.ai/ Collimator allows you to model, simulate, optimize, control, and collaborate in the cloud, with the power of Python and JAX New features: * Powered by JAX * Generative AI * Auto-Differentiation * PID Auto-Tune * SINDy model blocks * Model Predictive Control * Real-Time Collaboration * Hardware in the Loop * Hybrid models * State machines * FMU support * Updated ...
AI/ML+Physics Part 4: Crafting a Loss Function [Physics Informed Machine Learning]
Переглядів 27 тис.3 місяці тому
This video discusses the fourth stage of the machine learning process: (4) designing a loss function to assess the performance of the model. There are opportunities to incorporate physics into this stage of the process, such as adding regularization terms to promote sparsity or extra loss functions to ensure that a partial differential equation is satisfied, as in PINNs. This video was produced...
AI/ML+Physics Part 3: Designing an Architecture [Physics Informed Machine Learning]
Переглядів 34 тис.3 місяці тому
This video discusses the third stage of the machine learning process: (3) choosing an architecture with which to represent the model. This is one of the most exciting stages, including all of the new architectures, such as UNets, ResNets, SINDy, PINNs, Operator networks, and many more. There are opportunities to incorporate physics into this stage of the process, such as incorporating known sym...
AI/ML+Physics Part 2: Curating Training Data [Physics Informed Machine Learning]
Переглядів 26 тис.3 місяці тому
This video discusses the second stage of the machine learning process: (2) collecting and curating training data to inform the model. There are opportunities to incorporate physics into this stage of the process, such as data augmentation to incorporate known symmetries. This video was produced at the University of Washington, and we acknowledge funding support from the Boeing Company %%% CHAPT...
AI/ML+Physics Part 1: Choosing what to model [Physics Informed Machine Learning]
Переглядів 68 тис.4 місяці тому
This video discusses the first stage of the machine learning process: (1) formulating a problem to model. There are lots of opportunities to incorporate physics into this process, and learn new physics by applying ML to the right problem. This video was produced at the University of Washington, and we acknowledge funding support from the Boeing Company %%% CHAPTERS %%% 00:00 Intro 04:51 Decidin...
Physics Informed Machine Learning: High Level Overview of AI and ML in Science and Engineering
Переглядів 207 тис.4 місяці тому
This video describes how to incorporate physics into the machine learning process. The process of machine learning is broken down into five stages: (1) formulating a problem to model, (2) collecting and curating training data to inform the model, (3) choosing an architecture with which to represent the model, (4) designing a loss function to assess the performance of the model, and (5) selectin...
Can we make commercial aircraft faster? Mitigating transonic buffet with porous trailing edges
Переглядів 9 тис.5 місяців тому
Can we make commercial aircraft faster? Mitigating transonic buffet with porous trailing edges
A Neural Network Primer
Переглядів 35 тис.5 місяців тому
A Neural Network Primer
Supervised & Unsupervised Machine Learning
Переглядів 23 тис.5 місяців тому
Supervised & Unsupervised Machine Learning
A Machine Learning Primer: How to Build an ML Model
Переглядів 43 тис.5 місяців тому
A Machine Learning Primer: How to Build an ML Model
Arousal as a universal embedding for spatiotemporal brain dynamics
Переглядів 25 тис.6 місяців тому
Arousal as a universal embedding for spatiotemporal brain dynamics
New Advances in Artificial Intelligence and Machine Learning
Переглядів 72 тис.6 місяців тому
New Advances in Artificial Intelligence and Machine Learning
Nonlinear parametric models of viscoelastic fluid flows with SINDy
Переглядів 6 тис.6 місяців тому
Nonlinear parametric models of viscoelastic fluid flows with SINDy
[5/8] Control for Societal-Scale Challenges: Road Map 2030 [Technology, Validation, and Transition]
Переглядів 10 тис.11 місяців тому
[5/8] Control for Societal-Scale Challenges: Road Map 2030 [Technology, Validation, and Transition]
[8/8] Control for Societal-Scale Challenges: Road Map 2030 [Recommendations]
Переглядів 6 тис.11 місяців тому
[8/8] Control for Societal-Scale Challenges: Road Map 2030 [Recommendations]
[2/8] Control for Societal-Scale Challenges: Road Map 2030 [Societal Drivers]
Переглядів 7 тис.11 місяців тому
[2/8] Control for Societal-Scale Challenges: Road Map 2030 [Societal Drivers]
[1/8] Control for Societal-Scale Challenges: Road Map 2030 [Introduction]
Переглядів 15 тис.11 місяців тому
[1/8] Control for Societal-Scale Challenges: Road Map 2030 [Introduction]
[6/8] Control for Societal-Scale Challenges: Road Map 2030 [Education]
Переглядів 3,4 тис.11 місяців тому
[6/8] Control for Societal-Scale Challenges: Road Map 2030 [Education]
[3/8] Control for Societal-Scale Challenges: Road Map 2030 [Technological Trends]
Переглядів 5 тис.11 місяців тому
[3/8] Control for Societal-Scale Challenges: Road Map 2030 [Technological Trends]
[7/8] Control for Societal-Scale Challenges: Road Map 2030 [Ethics, Fairness, & Regulatory Issues]
Переглядів 2,1 тис.11 місяців тому
[7/8] Control for Societal-Scale Challenges: Road Map 2030 [Ethics, Fairness, & Regulatory Issues]
[4/8] Control for Societal-Scale Challenges: Road Map 2030 [Emerging Methodologies]
Переглядів 4,2 тис.11 місяців тому
[4/8] Control for Societal-Scale Challenges: Road Map 2030 [Emerging Methodologies]
🤯🤯🤯 Integrating GPT-4 into Collimator.ai for Symbolic Modeling and Control** 🔥🔥🔥
Переглядів 19 тис.Рік тому
🤯🤯🤯 Integrating GPT-4 into Collimator.ai for Symbolic Modeling and Control 🔥🔥🔥

КОМЕНТАРІ

  • @blakhokisbak
    @blakhokisbak 19 годин тому

    As a Chemical Engineer that studied CFD in grad school turned Data Scientist, I absolutely love this and the fact that there is active research in the intersection of physics and AI.

  • @dharmik2
    @dharmik2 21 годину тому

    what id really mass energy and momentum created then how will gauss divergence theorem work?

  • @paveltolmachev1898
    @paveltolmachev1898 День тому

    This is awesome! I am so stocked about how they have found a 15D linear system with integer sparse off-diagonal matrix approximating Lorenz attractor with such an accuracy. It is a pure joy contemplating this mathematical elegance

  • @SmithnWesson
    @SmithnWesson День тому

    It's not just a log function that has levels. The numbers themselves have levels. exp(i t) = exp(i (t + 2 pi k)) for all integer k It seems to me that any complex valued function also has levels. Why is the Log function singled out as special? Is it because it has a singularity?

  • @cygtrafolia1007
    @cygtrafolia1007 День тому

    epic lighting. subscribed.

  • @husna3349
    @husna3349 День тому

    Your words of choice are easy to understand and make the lecture sounds fun! Thankyou! Will sit for my final this 29june pray for me🙏

  • @a.b3203
    @a.b3203 2 дні тому

    9:21 so you change the lower bound to 0 because the lowest value we can obtain from solving the integral is now 0, rather than negative infinity, since you've defined it to be that way using H(t)? Is that about right?

  • @Daniboy370
    @Daniboy370 2 дні тому

    You outdid yourself as usual

  • @pain4743
    @pain4743 2 дні тому

    Amazing, Than you

  • @pain4743
    @pain4743 2 дні тому

    Amazing, Thank you

  • @umithangoktolga3770
    @umithangoktolga3770 2 дні тому

    2024, still the best.

  • @osianshelley3312
    @osianshelley3312 3 дні тому

    Fantastic video! Do you have any references for the mathematics behind the continuous adjoint method?

  • @MathCuriousity
    @MathCuriousity 3 дні тому

    STEVE I ABSOLUTLY LOVE HOW SHARP AND CLEAN AND PACKED FULL OF INFO YET STILL DIGESTIBLE EVERYTHING IS!! You are my favorite teacher. I am a self learner and you are GOD MODE for that. PS: please do a video on why all power series are Taylor series (without borel heavy machinery if possible or by using borel but breaking everything down)!

  • @MathCuriousity
    @MathCuriousity 3 дні тому

    Why is EVERY power series a Taylor series (without having to use heavy analysis stuff I don’t understand)!?

  • @sebastianrada4107
    @sebastianrada4107 3 дні тому

    You know that it is going to be a great lecture series when Eigensteve is teaching you about eigenvalues and eigenvectors

  • @HansPeter-gx9ew
    @HansPeter-gx9ew 3 дні тому

    tbh understanding his videos is very difficult, IMO he explains badly. Like 14:14 is the first more complicated part and I don't really get what it is about. I wouldn't understand ResNet from his explanation either if I had no prior knowledge about it. He just assumes that I am some expert in math and DLs

  • @nuclearrambo3167
    @nuclearrambo3167 3 дні тому

    suposse we have a discrete time signal x[n]=exp(jwn) and it is periodic with N. then exp(jw(n+N))=exp(jwn) thus exp(jwN)=1. because 1=exp(2(pi)k) where k is an integer, equation w=2(pi)k/N must hold. if N is chosen to be pi ,which is not an integer, x[n] is not periodic. consequently, has infinitely many unique values. In addition, for x[n] to be periodic, w must be some multiple of pi (true when k and N are integers).

  • @nnanyereugoemmanuel8327
    @nnanyereugoemmanuel8327 3 дні тому

    I'm so much very grateful for these videos you make. Keep on the good work.

  • @mickwilson99
    @mickwilson99 4 дні тому

    This is so technically correct, and simultaneously so obtuse, that my intuition fuse has melted. Please consider redoing this as 3D pseudo visualizations of data subsets.

  • @pineppolis
    @pineppolis 4 дні тому

    tim cook?

  • @julijangrajfoner1730
    @julijangrajfoner1730 4 дні тому

    How can sigma be invertible when it's a rectangular matrix?

  • @user-oj9iz4vb4q
    @user-oj9iz4vb4q 4 дні тому

    This seems like you are changing your loss function not your network. Like there is some underlying field you are trying to approximate and you're not commenting on the structure of the network for that function. You are only concerning yourself with how you are evaluating that function (integrating) to compare to reality. I think it's more correct to call these ODE Loss Functions, Euler Loss Functions, or Lagrange Loss Functions for neural network evaluation.

  • @Melle-sq4df
    @Melle-sq4df 5 днів тому

    this video is on a different level of teaching, thanks

  • @Jorge-ls9po
    @Jorge-ls9po 5 днів тому

    Nice vid and looking forward to follow all the content. By the way, how physics informed ML differs from the field of system identification?

  • @RimHail
    @RimHail 5 днів тому

    THANK U SIR . THAT WAS EXACTLY WHAT I AM LOOKING FOR.

  • @aviaser
    @aviaser 5 днів тому

    Thank you Steve. Great explanation! Greets from Argentina!

  • @reik2006
    @reik2006 6 днів тому

    18:50 also setting Ki >> Kp for the PI control shows the oscillation around the reference due to the intergral term very nicely

  • @kqb540
    @kqb540 7 днів тому

    Can someone please edit the squeaks out. I was really hoping to watch the videos. But the squeaks are jarring.

  • @mostafarahmani1404
    @mostafarahmani1404 7 днів тому

    Thanks Prof. Brunton, I really like your videos. My question is what if we have the system's data but it is not the impulse experiment. It is a combination of step response and chirp response. What should we do? I don't have access to the system anymore. Any input is appreciated.

  • @Daniboy370
    @Daniboy370 8 днів тому

    Amazing as usual

  • @ruhulhowlader716
    @ruhulhowlader716 9 днів тому

    Professor please show me that when a unit mass as a wave propagate and transfer energy to the mass energy is kept constant. I can find particle velocity and shear strain for a shear wave and the displacement at a particular point for any time t but I don’t get the total energy of at the point does not main the same value. As shear strain is directly related to the particle velocity, is it that I have to consider either particle velocity or shear strain plus displacement related velocity in the perpendicular direction of displacement. Please help me.

  • @Daniboy370
    @Daniboy370 9 днів тому

    You have an impressive ability to simplify complex subjects

  • @culturemanoftheages
    @culturemanoftheages 9 днів тому

    Excellent explanation! For those interested in LLMs residual connections are also featured in the vanilla transformer block. The idea is similar to CNN ResNets, but instead of gradually adding pixel resolution each block adds semantic "resolution" to the original embedded text input.

  • @maksymriabov1356
    @maksymriabov1356 9 днів тому

    IMHO you should speak a little faster and make less jests; for scientists watching this it wastes a time and attention.

    • @chrisnoble04
      @chrisnoble04 9 днів тому

      You can always run it at 2x speed....

  • @zealot4325
    @zealot4325 9 днів тому

    Thank you a lot!

  • @Ykotb08
    @Ykotb08 9 днів тому

    amazing as usual, I wish I had discovered this channel long time ago, control systems is my favorite field and this channel made me even love it more, thank you for clear explanation!

  • @davidmccabe1623
    @davidmccabe1623 9 днів тому

    Does anyone know if transformers have superseded resnets for image classification?

    • @culturemanoftheages
      @culturemanoftheages 9 днів тому

      Vision transformer (ViT) architectures have been studied that outperform CNN-based approaches in some respects, but they require more training data, more resources to train, and in general yield a bulkier model than a CNN would. They also use a different information-concentrating mechanism (attention for transformers vs. convolution for CNNs), so I imagine there are certain vision applications where transformers might be preferable.

  • @-mwolf
    @-mwolf 9 днів тому

    Awesome video. One question I'm asking myself is: Why isn't everybody using NODEs instead of resnets if they are so much better?

  • @physicsanimated1623
    @physicsanimated1623 9 днів тому

    Hi Steve - this is Vivek Karmarkar! Thanks for the video - great content as usual and keeps me motivated to create my own PINN content as well. Looking forward to the next video in the series and would love to talk PINN content creation with you! I have been thinking about presenting PINNs with ODEs as examples and its nice to contrast it with Neural ODEs - nomenclature aside, it looks like the power of the NN as universal approximators allows us to model either the flow field (Neural ODEs) or the physical field of interest (PINNs) for analysis which is pretty cool!

  • @lorisdemuth374
    @lorisdemuth374 9 днів тому

    Many thanks for the extremely good videos. Really well explained and easy to understand. A video on "Augmented neural ODEs" would go well with "neural ODEs" 😊

  • @ultrasound1459
    @ultrasound1459 9 днів тому

    ResNet is literally the best thing happened in Deep Learning.

  • @zack_120
    @zack_120 9 днів тому

    9:27- intro another entity here really messed things up

  • @zack_120
    @zack_120 9 днів тому

    RtoL writing behind a glass is revolutionizing online teaching, the 1st of which seems to be that on Nancy's channel on calculus. Super cerebrum-cerebellum-hand axis👍

  • @mostafasayahkarajy508
    @mostafasayahkarajy508 9 днів тому

    Thank you very much for your videos. I am glad that besides the classical sources to promote science (such as books and papers), your lectures can also be found on youtube. In my opinion, Prof. Bruton is the best provider of youtube lectures and I don't want to miss any of the lectures.

  • @ramimohammed3132
    @ramimohammed3132 10 днів тому

    thank u sire!

  • @cieciurka1
    @cieciurka1 10 днів тому

    SHAMEEEEEE🎉 Bound, border, infinity, noninfinity, natural, where is the end?! calc machine how it works, integer, costs money costs profits cons in mathematics, NOMIA! ECO? algorithm accuracy, fuzzy logic, integer 0-I. ONE BOOK NO INDIVIDUAL HERE 🎉WHEN YOU SMOOTHING GRADIENT YOU LOSING

  • @sainissunil
    @sainissunil 10 днів тому

    Thank you for making this. I watched your video on Neural ODEs before I watched this. It is much easier to understand the Neural ODE video now that I have watched this. I would love to watch a video about the ResNet classifier idea you discuss here. If you have already done that please add a link here. Thanks, and this is awesome!

  • @saraiva407
    @saraiva407 10 днів тому

    Thank you SO MUCH prof. Steve!! I intend to study neural networks in my graduate courses thanks to your lectures!! :D

  • @cieciurka1
    @cieciurka1 10 днів тому

    STEVE MAKE TWO. SMALLER HIGHER LIKE ARRAY ONE DIRECTION OR SYMMETRY LIKE MIRROR. FEEDBACK AND THIS 150.000ageSCIENCE.

  • @goodlack9093
    @goodlack9093 10 днів тому

    Thank you for this content! Love your approach. Please never stop educating people. We all need teachers like you!:) ps Enjoying reading your book