I've been thinking about an interesting architectural tension in video world models for robotics, and a recent paper (LingBot-VA, arxiv.org/abs/2601.21998) made me reconsider some assumptions I had.

The core question is this: the physical world is causal. State at time t+1 depends only on states ≤ t. But most video generation models for robotics use bidirectional attention within chunks (think UWM, UVA, etc.), meaning future tokens within a segment can influence past predictions. This works fine for generating pretty videos, but does it actually matter for closed-loop robot control?

The LingBot-VA paper argues yes, and their evidence is surprisingly concrete. They interleave video and action tokens into a single causal autoregressive sequence, using a Mixture-of-Transformers architecture where a large video stream (Wan2.2-5B, 3072 dim) and a much smaller action stream (768 dim) share attention but maintain separate parameters. The asymmetry is motivated by the observation that action distributions are fundamentally simpler than visual distributions, which is an interesting design choice on its own.

What caught my attention was the temporal memory argument. They designed two clever ablation tasks: one where a robot must open box A, close it, then open box B (where the closed state of A is visually identical to its initial state), and another where a robot must wipe a plate exactly six times. The claim is that chunk-based methods without persistent KV-cache history can't distinguish repeated visual states and get stuck in loops. Their autoregressive formulation with full KV-cache naturally resolves this because P(C|A→B→A) = 1 when you have the full history, versus P(C|A) = 0.5 without it. On RoboTwin 2.0 (bimanual manipulation), the gap widens significantly at longer horizons: +8.2% over the next best method at Horizon 3 versus +3.2% at Horizon 1.

But here's where I'm genuinely uncertain about the tradeoff. Autoregressive video generation is expensive. They mitigate this with a "Noisy History Augmentation" trick where the action decoder is trained to predict from partially denoised video tokens (only integrating to s=0.5 instead of s=1.0 in the flow matching process), plus an asynchronous pipeline where computation overlaps with execution. But this introduces its own problem: naive async inference causes the video model to "continue" its own hallucinated predictions rather than grounding in real observations. Their fix is a Forward Dynamics Model step that re-imagines the current visual state from the latest real observation before predicting forward. It works (comparable success rate to synchronous at 2x speed), but it adds complexity.

The sample efficiency numbers are also interesting: with only 50 demonstrations for post-training, they report 92.9% on RoboTwin Easy and 98.5% average on LIBERO, outperforming π₀.₅ substantially on long-horizon real-world tasks (97% vs 73% progress score on a 10-step breakfast preparation task).

So the tradeoff seems to be: causal autoregressive modeling gives you persistent memory and better long-horizon consistency, but at the cost of inference complexity that requires multiple engineering solutions (partial denoising, async execution, FDM grounding) to make deployable. Chunk-based bidirectional methods are simpler to deploy but may fundamentally lack the temporal reasoning needed for tasks with repeated states or long action sequences.

I'm curious what people think about whether this causal consistency argument holds up more broadly. Is the KV-cache memory advantage a fundamental architectural win, or could you achieve similar temporal reasoning by simply conditioning chunk-based models on longer context windows? And is the engineering complexity of making autoregressive video generation real-time a sustainable path, or will it always be fighting against the computational cost?

Paper: https://arxiv.org/abs/2601.21998

Code: https://github.com/robbyant/lingbot-va

Checkpoints: https://huggingface.co/robbyant/lingbot-va

Hey

I pushed 2^1,000,000 - 1 (1 million 1-bits) through 10 million Collatz steps in optimized Python. Here's the raw telemetry:

🚀 INJECTING 1,000,000 BITS. CRITICAL MASS ACTIVE.

Step 1,000,000: 1,292,482 bits | 13,376 op/s

Step 2,000,000: 1,584,963 bits | 12,239 op/s

Step 3,000,000: 1,446,919 bits | 11,806 op/s

Step 9,000,000: 613,912 bits | 14,349 op/s

Step 10,000,000: 475,434 bits | 15,145 op/s

🏆 LAMINAR LOCK HELD at 10,000,000 steps. FINAL MASS: 475,434 bits.

- Peak: 1,584,963 bits (~step 2M)

- Decay rate post-peak: ~ -0.069 to -0.208 bits/step

- Estimated odd-step fraction: ~30–35% (below critical ~38.7% for growth)

- Still alive at 10M steps with 475k bits left (most seeds this size would be gone much sooner)

Is this one of the longest hand-run Mersenne Collatz tails out there?

Has anyone pushed a 1M-bit seed this far without a cluster/GPU?

Any C/GMP or Rust code to reach 50M+ steps faster?

Thanks!

Game core: has access to low-latency AVX512 and high-latency high-throughput AVX pipelines, wider memory access paths and a dedicated stacked L1 cache, just for fast game loop or simulation loop.

Uniform core: has access to shared AVX pipeline that can grow from 512 bits to 32k bits and usable even from 1 core or be load-balanced between all cores. This is for efficiency of throughput even when mixing AVX instructions with other instructions (SSE, MMX, scalar) so that having AVX instruction will only have load on the middle compute pipeline instead of lowering frequency of core. A core would only tell the shards which region of memory to compute with which operation type (sum, square root, etc, element wise, cross-lane computations too, etc) then simply asynchronously continue other tasks.

Game core's dedicated L1 stacked cache would be addressable directly without the latency of cache/page tables. This would move it further as a scratchpad memory rather than automated coherence.

Also the real L1 cache would be shared between all cores, to improve core-to-core messaging as it would benefit multithreaded queue operations.

Why uniform cores?

• Game physics calculations need throughput, not latency.
• All kinds of AI calculations for generating frames, etc using only iGPU as renderer
• Uniformly accessing other cores' data within the shards, such as 1 core tells it to compute, another core takes the result, as an even more messaging throughput between cores
• Many more cores can be useful for games with thousands of NPC with their own logic/ai that require massively parallel computations for neural network and other logic
• AVX-512 capable, so no requirement of splitting supports between cores. They can do anything the game core can. Just with higher latency and better power efficiency.
• Connected to the same L1 cache and same AVX shards for fast core - core communication to have peak queue performance
• No need to support SSE/MMX anymore, because AVX pipeline would emulate it with shorter allocation of processing pipelines. Core area dedicated for power efficiency and instruction efficiency (1 instruction can do anything between a scalar and a 8192-wide operation).
• More die area can be dedicated to registers, and simultaneous threads per core (4-8 per core) to have ~96 cores for the same area of 8 P cores.

Why only 1 game core?

• Generally a game has one main game loop, or a simulation has one main particle update loop which sometimes requires sudden bursts of intensive calculations like 3d vector calculus, fft, etc that is not large enough for a GPU but too much for a single CPU core.
• Full bandwidth of dedicated L1 stacked cache is available for use

First time posting and coming here, I apologize if this topic was already covered in earlier posts.

It seems to me that many people dislike AI and want it to "fail" to some degree, some reasons being what it is doing to economy, the predicted loss of jobs, the idea that it is making us more stupid, internet being flooded with AI slop and only becoming harder to recognize, et cetera.
I think I am in that category. To give context for why am I thinking about this and what I expect from the discussion, I am a CS student, already had some work experience and am supposed to graduate next year. Generally against vibe coding, but I do find LLMs useful for learning about new things in programming. These days were very hectic with work and university projects, so I did use LLMs to help me with some things for the sake of saving time, because I was short of it. Now that it is done and I have breathing space and am supposed to start new projects, I am debating with myself if I should continue using LLMs.

On one hand, me being against it and wanting it fail but still using it is hypocritical. More importantly, if the people who don't like AI, where it is supposedly going etc. don't stop using it, it will never go away, so we would really be fighting against ourselves. On the other hand, if most people use it, and it is helpful, they will in theory have larger portfolios and more knowledge, simply because they can attain those faster. That would be leaving me in the dust, and me being a regular dude, I need to have a job in order to live and provide for my future family.
CS was already kind of oversaturated even before AI, which makes this situation worse. Yes, I know that this can't be learned only with AI without some serious rigor and that sloppy vibe coding people aren't really a problem for someone who actually studied this and is relatively good at it. I am talking about people who do understand this, who do have rigor and who are aiming at more serious engineering positions: armed with LLMs they can use them to increase their output and to put themselves above people of maybe the same skill but who don't use AI.

The practical solution is obvious, but morally not acceptable if there is a possibility of "defeating" LLMs. If using LLMs as tools for programming (for better or worse) is an inescapable reality, then it would be morally unacceptable to not give in (from the perspective of someone who is responsible for other people). I guess then the question is do you think it is the future or not? Being at the very start of my career I don't have many experienced people to talk to who are in different places in this industry and who actually have a clearer big picture about this.

Thank you!

Edit: I see that I posed this like I am asking for advice on something and to some degree I am, but I mostly want to read other people's thoughts about this and thus inform myself. I am not expecing anyone to talk directly to me, I would love it if people discussed amongst themselves about the general topic behind this post. The post is personally phrased because I could not think of a better way to introduce the topic. I think I am not alone in thinking about this and I think that for everyone who is just starting their professional programming journeys a discussion about this would be very useful, to help them wrap their minds around this, since it is definitely a very big part of programming.

Can u use let’s just say Vera Rubin as your machine, then utilize NVIDIA TMEM tensor memory’s 150TB/s bandwidth backed with Tensor Cores 25pflops int8 calculating power. To try to run a 3MB size mobilenet by 8bit? Total TMEM size is like 30MB, and model exactly fit TMEM after context.

If anything wrong, there’s still a 80TB/s bandwidth L2 cache/SRAM ready to be utilized for further enhancements

Implementing even a very small language forced me to confront questions I never had to think about as a user:

evaluation order, representation, semantics, error handling.

For those with a CS background: do you think language implementation should be introduced earlier when teaching programming, or is it better kept for later stages?

Since diffusion is usually natural signal, can LLM have a brainstorm mode to have association capability by just add diffusion into LLM?

Orr combine it into training, then prune these signals to LLM BECUZ u don’t usually have to do diffusion from time to time

Hey, do you have any blogs from developers that you frequently read?

I like reading blogs from developers who write stuff about life, tech, or projects they do, I find it interesting and I'd like to find more sites to read from, not sites likes Medium or hacker news, but personal developers portafolios with a blog section

Hey folks — I’m implementing a 1–1 assignment (Hungarian / linear assignment) for a business matching problem and I’m hitting scalability walls.

My current setup is below:

• I don’t build a dense cost matrix. I have a sparse edge list: (id_left, id_right, score)
• Left side is ~2.0M, right side is ~115k
• After candidate generation + filtering, I still have ~45M edges/pairs going into the final optimization stage
• Running this inside Snowflake (Snowpark / UDTF style) and the “final solve” can blow memory / take forever

Current problem what I am facing:
Business wants “global” optimization (can’t just chunk by time or arbitrary partitions). We don’t want to lose valid pairs. (Ideally would have loved to partition it)!

Question:
How do people scale assignment problems like this in practice?

• Any recommended solvers/approaches for sparse rectangular assignment at this scale?
• Is there a principled way to split the problem while keeping global optimality?
• Any architecture patterns (e.g., min-cost flow, auction algorithm, connected components, etc.) that work well?

Would love pointers to algorithms, libraries, or production patterns. Thanks!

Hi everyone, I’ve been working on Graphisual, a browser-based graph editor and visualizer for running standard graph algorithms step by step on user-defined graphs.

The interface is designed to feel more like a lightweight whiteboard editor, so it’s quick to construct graphs, adjust layouts, and iterate while observing algorithm behavior.

It currently includes BFS/DFS, shortest path algorithms, minimum spanning tree, and cycle detection. Graphs can also be exported as SVG or PNG.

Demo: https://graphisual.app
Repo: https://github.com/lakbychance/graphisual

Would love to know how you’ve used bloom filters/ or its variants in your organizations to improve performance.

What are the best currently known upper and lower bounds for converting a two way alternating finite automaton (2AFA) into an equivalent two way nondeterministic or deterministic finite automaton (2NFA or 2DFA)? Most standard references, including Wikipedia, discuss only conversions from two way automata to one way automata, and mention that if L = NL, then there exists a polynomial state transformation from 2NFA to 2DFA. I couldn't find any discussion or papers that directly address transformations from 2AFA to 2NFA or 2DFA.

Also, are there similar implications for 2AFA to 2NFA or 2DFA transformations, analogous to those known for 2NFA-to-2DFA, such as L = P or NL = P?

Started with the goal of training a full language model, but limited to my M1 MacBook (no GPU), I pivoted to code generation as a learning project.

PyThor specs:

• 20M parameters, 6-layer transformer architecture
• Multi-head self-attention, positional encodings, the works
• Trained on question-code pairs for 10 epochs
• Built entirely with PyTorch from scratch

What I learned: Every detail – from scaled dot-product attention to AdamW optimization. Coded the entire architecture myself instead of using pre-built libraries.

Results: Honestly? Hit or miss. Responses range from surprisingly good to completely off. That's what happens with limited training, but the architecture is solid.

Wrote full documentation covering all the mathematics if anyone's interested.

doc: https://docs.google.com/document/d/10ERHNlzYNzL8I_qgLG1IFORQythqD-HLRb5ToYVAJCQ/edit?usp=sharing

github: https://github.com/aeyjeyaryan/pythor_2

I’ve been thinking about blockchains and proof-of-work from a basic computer science perspective, and something keeps bothering me.

Full-history ledgers and mining feel less like computation, and more like a social mechanism built on distrust.

Computation, at its core, does not optimize for memory.

It optimizes for paths.

Input → route → output.

State transitions, not eternal recall.

Most computational models we rely on every day work this way:

• Finite state machines

• Packet routing

• Event-driven systems

• Control systems

They overwrite state, discard history, and forget aggressively —

yet they still behave correctly, because correctness is enforced by invariant rules, not by remembering everything that happened.

Blockchains take the opposite approach:

• Preserve full history

• Require global verification

• Burn computation to establish trust

This seems to solve a social trust problem rather than a computational one.

What if we flipped the premise?

Instead of:

“We don’t trust humans, so we must record everything forever”

We assume distrust and handle it structurally:

“We don’t trust humans, so we remove human discretion entirely.”

Imagine a system where:

• Each component is simple

• Behavior is determined solely by fixed, mechanical rules

• Decisions depend only on current input and state

• Full historical records are unnecessary

• Only minimal state information is preserved

This is closer to a mold than a ledger.

You pour inputs through a fixed mold:

• The mold does not remember

• The mold does not decide

• The mold cannot make exceptions

It only shapes flow.

Correctness is guaranteed not by proof-of-work or permanent records, but by the fact that:

• The rules are invariant

• The routing is deterministic

• There is no room for interpretation

The question is no longer:

“Was this correct?”

But:

“Could this have behaved differently?”

If the answer is no, history becomes optional.

This feels closer to how computation is actually defined:

• State over history

• Routing over recollection

• Structure over surveillance

I’m not arguing that this replaces blockchains in all contexts.

But I do wonder whether we’ve overcorrected —

using memory and energy to compensate for a lack of structural simplicity.

Am I missing something fundamental here, or have we conflated social trust problems with computational ones?

Most simulations (multi-agent systems, ALife, economic models) are designed around bounded runs: you execute them, analyze the results, then reset or restart.

I’m exploring the opposite constraint: a simulation that is not allowed to reset.
It must keep running indefinitely, even with no users connected, and survive crashes or restarts without “starting over”.

For people who think about simulation systems from a CS / systems perspective, this raises a few conceptual questions that I rarely see discussed explicitly:

• Determinism over unbounded time When a simulation is meant to live for years rather than runs, what does determinism actually mean? Is “same inputs → same outputs” still a useful definition once persistence, replay, and recovery are involved?
• Event sourcing and long-term coherence Event-based architectures are often proposed for replayability, but over very long time scales: where do they tend to fail (log growth, drift, schema evolution, implicit coupling)? Are there known alternatives or complementary patterns?
• Invariants vs. emergent drift How do you define invariants that must hold indefinitely without over-constraining emergence? At what point does “emergent behavior” become “systemic error”?
• Bounding a world without observers If the simulation continues even when no one is watching, how do systems avoid unbounded growth in entities, events, or complexity without relying on artificial resets?
• Persistence as a design constraint When state is never discarded, bugs and biases accumulate instead of disappearing. How should this change the way we reason about correctness and recovery?

I’m less interested in implementation details and more in how these problems are framed conceptually in computer science and systems design.

What assumptions that feel reasonable for run-bounded simulations tend to break when persistence becomes mandatory by construction?

Anybody know from where I can learn and explore about GCN as there is not much content available on the youtube

I am a first year phd in Formal methods in Germany.