888Flux
kiwifarms.net
- Joined
- Jul 6, 2021
That would be an interesting concept. "Closed-book" models that can't make use of new information are going to be incredibly limited and not commercially viable in any capacity. Here is paper from DeepMind that goes into detail on self-learning through the use of new information that the model collects from Google search https://deepai.org/publication/inte...-prompting-for-open-domain-question-answeringBaking everything into a language model isn't the way to go about this. There's a clue in the name.
A better approach would be using multi-layered language models to structure and interface with a knowledge base (probably meta-organised with symbolic tree shit I won't bother going into). I've been thinking about this a bit lately.