Random notes on experiments with large language models. ### 2025-05-24 #### Devstral This model claims to be a highly performant coding agent. I asked it to write a Fibonacci function with this prompt: ``` >>> write a highly performant python function that calculates fibonacci. be succinct, concise with few comments unless necessary. ``` with the following output: Here's a highly performant Python function using memoization to calculate Fibonacci numbers: ```python def fib(n, memo={}): if n in memo: return memo[n] if n <= 2: return 1 memo[n] = fib(n-1, memo) + fib(n-2, memo) return memo[n] ``` This function uses a dictionary to store previously calculated Fibonacci numbers, significantly improving performance for large inputs. ### 2025-05-25 #### Openhands [All-Hands](https://docs.all-hands.dev) - Tried this out locally with Devstral. Its like a slightly slower chatgpt/claude. I asked it a few questions like building a Fibonacci function or asking it what Python's `id` is used for. #### Perceptrons ... #### Sigmoids σ - the sigmoid ``` Incidentally, σ is sometimes called the _logistic function_, and this new class of neurons called _logistic neurons_. It's useful to remember this terminology, since these terms are used by many people working with neural nets. However, we'll stick with the sigmoid terminology. ``` What about the algebraic form of σ? How can we understand that? In fact, the exact form of σ isn't so important - what really matters is the shape of the function when plotted. Here's the shape: ![[Pasted image 20250526055131.png]] This shape is a smoothed out version of a step function: ![[Pasted image 20250526055153.png]] ##### Architecture of a Neural Network The leftmost layer in this network is called the input layer, and the neurons within the layer are called _input neurons_. The rightmost or _output_ layer contains the _output neurons_, or, as in this case, a single output neuron. The middle layer is called a _hidden layer_, since the neurons in this layer are neither inputs nor outputs: ![[Pasted image 20250526055559.png]] Neural networks where the output from one layer is used as input to the next layer are called feedforward neural networks. This means there are no loops in the network - information is always fed forward, never fed back. ##### Exercises for NN and Deep Learning [Book](http://neuralnetworksanddeeplearning.com) - **Sigmoid neurons simulating perceptrons, part I** Suppose we take all the weights and biases in a network of perceptrons, and multiply them by a positive constant, `c>0`. Show that the `behaviour` of the network doesn't change. - **Sigmoid neurons simulating perceptrons, part II** Suppose we have the same setup as the last problem - a network of perceptrons. Suppose also that the overall input to the network of perceptrons has been chosen. We won't need the actual input value, we just need the input to have been fixed. Suppose the weights and biases are such that w⋅x+b≠0 for the input x to any particular perceptron in the network. Now replace all the perceptrons in the network by sigmoid neurons, and multiply the weights and biases by a positive constant c>0. Show that in the limit as c→∞ the behaviour of this network of sigmoid neurons is exactly the same as the network of perceptrons. How can this fail when w⋅x+b=0 for one of the perceptrons?