Home on Sid Mani's Blog
https://sidmani.com/
Recent content in Home on Sid Mani's BlogHugo -- gohugo.ioen-usWed, 28 Apr 2021 12:18:59 -0700Deciding the behavior of edited programs
https://sidmani.com/posts/behavior-of-edited-programs/
Wed, 28 Apr 2021 12:18:59 -0700https://sidmani.com/posts/behavior-of-edited-programs/I’ve been thinking about computation and program synthesis a lot recently. Program synthesis is the process of constructing a program based on constraints or example input/output pairs. This problem is extraordinarily difficult; the best methods essentially enumerate all reasonable programs, for some heuristic definition of reasonable, and run them to see if they work. In this situation, it would be convenient if, knowing what a program does, you could make a guess at what an edited version of the program does.Enumerating programs that halt
https://sidmani.com/posts/enumerating-programs-that-halt/
Fri, 16 Apr 2021 18:47:27 -0700https://sidmani.com/posts/enumerating-programs-that-halt/Suppose there’s a binary string, and we want to find a program that generates the string. Further assume that we are in possession of a universal Turing machine, that accepts programs and input in a predetermined binary representation. Then, how do we find such a program? Naively searching (by evaluating all length-\(n\) programs for all \(n\)) is dangerous, since we cannot know if any particular program halts; our procedure might get stuck while evaluating some program.Connecting the halting problem and induction
https://sidmani.com/posts/halting-problem/
Sun, 11 Apr 2021 12:38:05 -0700https://sidmani.com/posts/halting-problem/Suppose we have a program \(\mathbf{H}(p, x)\) that solves the halting problem for any program \(p\) evaluated on input \(x\), returning \(\mathrm{True}\) if the program halts and \(\mathrm{False}\) otherwise. Consider statements of the form \(\forall n \in \mathbb{N}: A(n)\), where \(A\) is some proposition that can be easily evaluated for any particular natural number. Let \(p(A)\) be the following program:
i = 1 loop forever: if not A(i): return i += 1 Then the evaluation of \(\neg\mathbf{H}(p, A)\) constitutes a proof of the truth or falsehood of the statement \(A\) for all natural numbers, if the evaluation of \(A(n)\) is guaranteed to halt.Backpropagation, hierarchy, and adversarial examples
https://sidmani.com/posts/backprop/
Sun, 28 Mar 2021 19:26:00 -0700https://sidmani.com/posts/backprop/One could easily believe that backpropagation ranks among the three most-computed algorithms in history.1 Backpropagation (backprop) exploits the chain rule from multivariable calculus to enable optimization of a parameterized function, usually a neural network.2 In order to support calculus, the function must be differentiable everywhere, and therefore continuous. We’ve already taken a look at the dangers of continuity. In this post, we’ll examine how the mechanics of backprop could harm representation learning.Difficulties with continuous approximation
https://sidmani.com/posts/difficulties-continuous-approximation/
Sat, 20 Mar 2021 20:32:59 -0700https://sidmani.com/posts/difficulties-continuous-approximation/Deep learning with artificial neural networks has several serious limitations that, in my opinion, render it completely incapable of modeling the kind of intelligence that humans possess. In the last few years, deep learning has seen several huge successes: GPT-3, AlphaGo, AlphaFold 2. These achievements were obtained through clever architectures based mostly on artificial neural networks and trained with massive computational scale.
The scaling laws diagram from the OpenAI team has frequently been presented as empirical proof of the power of large models:What are shapes?
https://sidmani.com/posts/what-are-shapes/
Wed, 24 Feb 2021 16:54:12 -0800https://sidmani.com/posts/what-are-shapes/I’ve been studying geometric representations and computer graphics for some time now, and I haven’t seen anyone talk about shapes and geometry in an accessible way. So in this post, I’ll quickly go over what shapes really are, and where they come from.
A shape is an object with volume and a surface. There are all sorts of technicalities that arise when definining volumes and surfaces, involving self-intersections, holes, and edges.A glossary of Japanese self-study
https://sidmani.com/posts/glossary-of-japanese/
Thu, 11 Jul 2019 11:27:46 -0800https://sidmani.com/posts/glossary-of-japanese/Japanese is arguably the most difficult language for English speakers to learn,1 yet the relatively recent popularization of Japanese media has resulted in a disproportionate number of learners on the Internet. Consider the number of subscribers to the various language-learning subreddits,2 which puts Japanese at more than two times as popular as the next-largest language, French. The explosive growth of the Japanese-learning community has produced a large number of free online resources, albeit of varying quality, and beginners are likely to be completely overwhelmed.