The team from Pathway believes that the transformer architecture developed eight years ago has reached its limits, which no amount of computing can overcome. Nor is there the power available to sustain itself. Further, there’s a lack of temporal reasoning and continual learning.
And so, as discussed in a research paper Pathway published in September and our interview at AWS re:Invent, the company is building a post-transformer-era frontier model built on neuronal dynamics with dragons in mind — a Dragon Hatchling architecture, inspired…
