[Me & X] A judgement system for decisions
X : How do you use AI?
Me : I assume you mean LLM/GPTs ... in which case, oodles of different ways. What are you after?
X : Important decisions?
Me : For quite some time, I've tinkered with and run a judgement system.
X : ?
Me : It's fairly simple, every question I ask is turned into a thesis, the counter is created (antithesis). Two agents then take on those roles and the case is argued through several rounds (minimum of three, maximum of ten). A group of 12 agents then vote (with public reasoning) after each round - the first three rounds are merely indicative and there's also a zero round vote on the quality of the thesis / antithesis.
A judging agent then decides at the end of each vote whether the arguments are materially different and if there has been a successful conclusion. Without a successful conclusion then the game continues (again there must be at least 3 rounds). Both the arguing agents have access to the argument, the counters, the voters comments and votes. Each round they present a refined argument. A court recorder summaries the thesis, antithesis, the main arguments presented and which argument eventually wins (if any does).
I then read it. The idea came from a conversation I had with Justine Tunney many years ago. Smart cookie that one.
X : And?
Me : Well, I often use multiple different LLMs, some local, some provided by online services. It's fascinating to see some of the biases that come in.
X : But do you use it to make decisions?
Me : I use it to inform myself of things I might not have considered otherwise but warrant investigation.
X : Seems like a lot of work.
Me : Good questions are hard to find.
X : But I'm looking for something that can give me answers.
Me : I know. Alas, we don't have thinking machines. Best we've got are swarms of different biases you can use to extract interesting questions from combined with statistical answers to very tightly constrained and understood problems.
X : Can I have access to your judgement system?
Me : No.
X : Why?
Me : You'll just misuse it to give you answers. It is designed to give you questions. That's actually the really interesting part.
X : When are you going to release it?
Me : Probably never. Large chunks are vibe coded and I wouldn't let anything that I perceive could cause harm and I don't fully comprehend the code of outside of my own firewall. It's just another of my own personal experiments / prototypes.
X : You haven't answered my question though.
Me : Oh, I have. For important decisions, seek alternative questions and use your brain.
Originally published on LinkedIn.