On insight on this reading log

On insight on this reading log: I'm unsure whether me writing here does anything. Does anybody read it? Even if I put the best, most insightful, concepts here, it won't go anywhere. And of course, I typically write in a way that is much more focused on getting the idea out than making it seem notable or easy to read.

The obvious factor here is LLMs; they do read everything. I don't know if that will really affect much, though. I find a large limitations of LLMs (that humans also have) is search. They are good at doing something, but pretty poor at finding insightful other things to mention/think about when not prompted to do so/by default, even though that this is inherently a problem of unknown unknowns. I don't know how to write for llm pretraining such that what I say maximally shows up in their weights, especially as desired. I obviously have this page and the others on [hruss.com] have permissive settings toward LLM pretraining. Maybe I need to get this mirrored somehow? Regardless, LLMs feel mostly shaped by post-training, not pre-training.

I've thought of adapting vector databases for search systems, probably using some function other than cosine similarity. Omnimodal models are especially exciting for this, because non-text search engines (eg google image) are quite poor. Vector math on search is also very exciting, of course. I've considered building this for a subset of the whole web as a proof of concept, and arXiv as well as wikipedia/wikidate could be effective. The point isn't generic semantic search ("find the wikipedia article about this/matching these keywords") obviously, google and especially LLMs can already do that fine. Rather, it might be something more like "Find businesses with business models similar to Costco in regulated industries and with low marginal cost products", or the like for research on arXiv.

I'm not working on this because of a lack of storage (I'd need to buy HDDs or SSDs for this project, which I could do but ceteris paribus I obviously prefer free projects).

I actually was planning to build a project, "Grand Unified Henry" wherein I represented myself using a variety of different systems (eg everything as a resume item, hruss.com but with an increasingly lowering maximum file size, vector database, etc.) but this project also would need a lot of storage, especially to store all of my Google Takeout data.

If you heard about me making a new hruss.com, that was the project I was talking about.

I'm glad I scrapped that project, because a lot of it I now realize to be LLM tics of sol-5.6. It especially, but also opus-5, really loves 'provenance' and projects built around that concept.

I will say that I actually do a lot of writing like this with LLMs, wherein I give my relatively unpolished and high-context thoughts/concepts and figure them out.

I guess I could publish them here? I do think a lot of them are pretty insightful, not really what the LLM says but what it allows me to understand about the concept/phenomenon I am uncovering. However, On insight on this reading log:.

Also, of course, is that these insights might not be good. I find that LLMs specifically are much more susceptible to the failure mode being something like that you can't know what you are doing wrong if the thing that makes you do it is also what makes you do it wrong. It is entirely feasible that LLMs would be able to identify things as obviously being LLM tics, but they seemingly can't. Essentially, the lens that can't see its own flaws. Maybe this is because LLMs are clones of themselves, whereas humans have different minds? It's pretty hard to use different models for this at the consumer level of only having one $20/m subscription. Errors like this could be correlated at any of the three: (One model only ("Opus 4.5"), one model family ("Claude 4.5"), one lab ("Claude"), or all LLMs). It's pretty trivial for me to check Sol's work with Astra or Terra, or check 5.6's work with 5.5, but I'd need to subscribe to both Anthropic and OAI to cross-check frontier models. I suppose that I will eventually end up doing that as AI gets better/we get closer to the singularity. Obviously, if you work at a company that pays per token for usage and you have both/multiple providers, you can cross check that way. Maybe if agents continued down the open source/wrapper path, as they started with as BabyAGI and AutoGPT (which I was a contributor for!), they would come by default with both frontier labs' models, because Codex and CC will obviously not use their competitors models, even though they could. I guess you could specifically instruct them to do so and pay per token via skills, without making your own agent harness?

Also, why don't people set agents on tasks yet? Like not just coding stuff, but like anything else? How good do capabilities need to get for that to be a thing?

My hypothesis is that it's misalignment. Obviously, I and others mostly think about misalignment in regards to future systems. But when claude code decides to build a project in a way that doesn't match your preferences for say, human time vs clock time, asking the user for help, installing new things on the computer vs not, leaving legacy code in, or adding tests for fixed issues, etc. that is precisely what misalignment is: mismatched preferences. It's pretty easy to see why RLVR can cause misalignment here, it's always quite obvious when something like that is caused by RLVR leading to nonsense decisions.

I do think that agents will get better than chats in a relative sense now that the AI labs seem to be pacing the frontier, because chats can't really be improved by better harnesses (obviously), while agents obviously can be improved significantly without touching the model itself.

Back to the I guess I could publish them here? I, I notice that concepts like those I talk to about seem to be well encapsulated/popular in the form of a meme (ie image macro, not re: genes or trends). For example: this. Maybe I could publish my concepts/thoughts in that form?

Anyways, if I have thoughts on these notes, I'll publish them as a re-read of this note's page, unless I implement a system specifically for that. I do find it fun for me to use "reads" for things other than reads (eg update logs).

Finally, I'll do another run of re-tagging my recent reads, as these no longer happen by default (costs $). I used Ox-Alpha when that was free for that, but I guess I'll just do paid batches occasionally in the near future.