Skip to content

Commit d33db08

Browse files
committed
selling points
1 parent c4fa3e3 commit d33db08

File tree

2 files changed

+29
-3
lines changed

2 files changed

+29
-3
lines changed

docs/source/figure/platonize.png

1.17 MB
Loading

docs/source/index.rst

Lines changed: 29 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,36 @@
11
Welcome to the Modula docs!
22
============================
33

4-
Modula is a deep learning framework and a deep learning theory built hand-in-hand. The central idea of Modula is to metrize the neural architecture and construct the corresponding duality theory. This is leading to training algorithms that are faster and more intrinsically scalable. But I anticipate other benefits too.
4+
Modula is a deep learning library and a deep learning theory built hand-in-hand. Modula disentangles complex neural networks and turns them into structured mathematical objects called *modules*. This makes training easier and also provides tools for understanding the properties of the trained network.
55

6-
Purpose of the docs
7-
^^^^^^^^^^^^^^^^^^^^
6+
.. image:: figure/platonize.png
7+
:align: center
8+
:width: 80%
9+
:class: no-scaled-link
10+
11+
Modula instantiates a set of theoretical ideas that I refer to as *metrized deep learning*. The central idea behind metrized deep learning is to equip all spaces inside a neural network with meaningful distance measures: this includes the activation spaces, the individual tensor spaces as well as the overall weight space. There are a few advantages to building neural networks in Modula:
12+
13+
.. grid:: 2
14+
:gutter: 3
15+
16+
.. grid-item-card:: Fast 🏎️
17+
18+
Modula automatically builds duality-based training algorithms: think Muon optimizer for any architecture.
19+
20+
.. grid-item-card:: Scalable 📈
21+
22+
Scaling is built directly into the fabric of the library, giving you learning rate transfer across various architectural dimensions.
23+
24+
.. grid-item-card:: Lipschitz (work-in-progress) ⛰️
25+
26+
Modula lets you train networks with automatically constructed Lipschitz certificates---in both inputs and weights.
27+
28+
.. grid-item-card:: Numerically sound 🧮
29+
30+
Modula helps you understand and control the basic numerical properties of your activations, weights and updates.
31+
32+
About these docs
33+
^^^^^^^^^^^^^^^^^
834

935
I'm currently in the process of overhauling these docs. But the idea is to create a central place to learn about the theory, algorithms and code behind Modula. I hope that this will help inspire further research into metrized deep learning.
1036

0 commit comments

Comments
 (0)