I'm assuming this is all based on a current snapshot in time and excludes any predictions of what may likely happen over the next 4 months. With that said, I would be interested in seeing this same exercise performed with ChatGPT and Claude every month leading up to the election to see how it changes and whether the original predictions were accurate. Great stuff!
I'd say that it's a bit more than a snapshot in time - which is actually what may make the Keys a better forecasting tool than polling. Many keys are becoming essentially locked in at this point leading to the election, e.g., party mandate, incumbency, primary challenger, charisma, scandals, etc. I'd say what really has potential to change is whether or not there is a third party challenger, i.e., if RFK melts away or remains near 10% polling, and social unrest significantly changes.
That's really interesting; I'd never heard of Lichtman's Keys before. It's even more interesting that the AI changed its answers to three questions when you gave it additional information to consider. If its job is to weigh variables it makes sense that it would adjust its evaluation based on new information. But that information was already out there, so why didn't it factor that in to its original answers?
It's strange that Gemini begged off of the very first question. All it had to do was say whether it was familiar with the Keys concept or not. (And it's funny/not funny that it offered Google Search as an alternative to "deliberately sharing something that's inaccurate." As opposed to deliberately sharing something that's curated to produce only what Google wants people to see. I don't use Google anyway but I recently switched to Brave. It might or might not be true that it doesn't spy as hard as the others, but its search results *suck.* When I want any kind of chance to find something specific I go back to Safari and use DDG. And I'm not talking about political stuff. I'm talking about *anything.* Brave produces the most insipid, irrelevant search results ever.)
Glad you enjoyed; it seems like a tool such as the Keys to the White House may be a more powerful longterm forecaster than polling. As James stated, polls are really just a snapshot at a given moment and are very sensitive to change day-by-day. This far out means nearly nothing in terms of predictive power. As for the AI changing it's mind, like you said this is information that it should have already contained in it's knowledge base. This makes me think that by me reminding it of this information, it changed a weight in it's consideration of these variables. I haven't decided if that's a good or bad thing though, i.e., can we just bias the AI into giving an answer we want by reminding it of specific factors?
You summarized this nicely - I am still one of those people who use google products like google search, gmail, and youtube. But I think that time may be coming to an end. I can't think of any nice way to say it - but Gemini has been and continues to be a dumpster fire. It is so unbelievably behind compared to nearly all other LLMs, despite the enormous bank account Google has compared to these other companies. I think they are so worried about politics at this point they are self censoring factual information.
I'm sorry to hear about Brave underperforming. My recommendation to you regarding search - try using ChatGPT as a search engine - it has impressed me more than once!
Haha the writing of this was quite a coincidence, considering the assassination attempt occurred later that same day. But I do think what you're describing would fall under 'charisma', which we already grant Trump.
I'm assuming this is all based on a current snapshot in time and excludes any predictions of what may likely happen over the next 4 months. With that said, I would be interested in seeing this same exercise performed with ChatGPT and Claude every month leading up to the election to see how it changes and whether the original predictions were accurate. Great stuff!
I'd say that it's a bit more than a snapshot in time - which is actually what may make the Keys a better forecasting tool than polling. Many keys are becoming essentially locked in at this point leading to the election, e.g., party mandate, incumbency, primary challenger, charisma, scandals, etc. I'd say what really has potential to change is whether or not there is a third party challenger, i.e., if RFK melts away or remains near 10% polling, and social unrest significantly changes.
That's really interesting; I'd never heard of Lichtman's Keys before. It's even more interesting that the AI changed its answers to three questions when you gave it additional information to consider. If its job is to weigh variables it makes sense that it would adjust its evaluation based on new information. But that information was already out there, so why didn't it factor that in to its original answers?
It's strange that Gemini begged off of the very first question. All it had to do was say whether it was familiar with the Keys concept or not. (And it's funny/not funny that it offered Google Search as an alternative to "deliberately sharing something that's inaccurate." As opposed to deliberately sharing something that's curated to produce only what Google wants people to see. I don't use Google anyway but I recently switched to Brave. It might or might not be true that it doesn't spy as hard as the others, but its search results *suck.* When I want any kind of chance to find something specific I go back to Safari and use DDG. And I'm not talking about political stuff. I'm talking about *anything.* Brave produces the most insipid, irrelevant search results ever.)
Glad you enjoyed; it seems like a tool such as the Keys to the White House may be a more powerful longterm forecaster than polling. As James stated, polls are really just a snapshot at a given moment and are very sensitive to change day-by-day. This far out means nearly nothing in terms of predictive power. As for the AI changing it's mind, like you said this is information that it should have already contained in it's knowledge base. This makes me think that by me reminding it of this information, it changed a weight in it's consideration of these variables. I haven't decided if that's a good or bad thing though, i.e., can we just bias the AI into giving an answer we want by reminding it of specific factors?
You summarized this nicely - I am still one of those people who use google products like google search, gmail, and youtube. But I think that time may be coming to an end. I can't think of any nice way to say it - but Gemini has been and continues to be a dumpster fire. It is so unbelievably behind compared to nearly all other LLMs, despite the enormous bank account Google has compared to these other companies. I think they are so worried about politics at this point they are self censoring factual information.
I'm sorry to hear about Brave underperforming. My recommendation to you regarding search - try using ChatGPT as a search engine - it has impressed me more than once!
Key #14 should be: Did the president survive an assassination and pump his fist immediately after?
Haha the writing of this was quite a coincidence, considering the assassination attempt occurred later that same day. But I do think what you're describing would fall under 'charisma', which we already grant Trump.
Fair enough! His charisma certainly earned him probably 2+ points in some states after an event like this.
Great experiment, the results will be verified for correctness in a few months. AI predicting the presidency is much stronger than human polls.