Checkpoint ai.md

_d/ai.md

@@ -8,44 +8,54 @@ A landing page for all my ai pages - a nice jumping off point (especially from t
 
 {%include summarize-page.html src="/ai-art" %}
 
----
-
 {%include summarize-page.html src="/ai-image" %}
 
----
-
 {%include summarize-page.html src="/ai-coder" %}
 
----
-
 {%include summarize-page.html src="/ai-paper" %}
 
----
-
 {%include summarize-page.html src="/ai-bestie" %}
 
----
-
 {%include summarize-page.html src="/ai-developer" %}
 
----
-
 {%include summarize-page.html src="/ai-security" %}
 
----
-
 {%include summarize-page.html src="/ai-journal" %}
 
----
-
 {%include summarize-page.html src="/ai-testing" %}
 
-Older stuf
+{%include summarize-page.html src="/ml" %}
 
----
+{%include summarize-page.html src="/recommend" %}
 
-{%include summarize-page.html src="/ml" %}
+### Ideas/Thoughts/Tidbits
 
----
+### Examples of Alchamey
 
-{%include summarize-page.html src="/recommend" %}
+- Doing worst job in dec, (and July for European models) - [crappy source](https://www.techradar.com/computing/artificial-intelligence/ai-might-take-a-winter-break-as-gpt-4-turbo-apparently-learns-from-us-to-wind-down-for-the-holidays)
+
+### How do LLMs think?
+
+- "Complete the next word" is super hard. Imagine a big long mystery novel, and then the next sentence is - And the killer was .... That requires a very deep understanding
+- LLMs currently are very "fast thinking" from the thinking fast and slow book.
+
+### LLM training efficiency
+
+- LLMs are trained very inefficiently. Imagine teaching a kid by making them read everything and see what happens most.
+- In the future rewards based on process , not outcome (e.g did I take a good first step, vs did I get the right answer).
+- We train LLMs based on hard/easy for humans, what about when we do it by stuff that is hard/easy for LLMs
+
+### It's not good enough today
+
+- Imagine saying that of a pre-school student.
+- In 2024, we have gpt-4o (high school student), 2 years ago, we had a pre-schooler.
+- What will we have in 2 more years?
+
+### Where will we get the OOMs - Order Of Magnitude Improvements
+
+See situational Awareness
+
+- Compute power: The essay mentions an increase of "~0.5 orders of magnitude or OOMs/year" in compute power - . This is driven by the rapid expansion of computing infrastructure, with plans for trillion-dollar compute clusters and hundreds of millions of GPUs being deployed across the United States.
+- Algorithmic efficiencies: Another "~0.5 OOMs/year" is expected to come from improvements in AI algorithms- . These advancements are likely to enhance the capabilities of AI systems significantly.
+- "Unhobbling" gains: LLMs have latent capabilities (for example when they use Chain Of Thought). By using that, we get a boost. How many more such boosts are there.
+- AI research automation: Once AGI (Artificial General Intelligence) is achieved, the essay suggests that "Hundreds of millions of AGIs could automate AI research, compressing a decade of algorithmic progress (5+ OOMs) into ≤1 year" - This rapid acceleration in AI capabilities could lead to a dramatic increase in overall intelligence.