Jack of some trades.
Problems I’m working on currently:
Decis Research Report Good morning and Happy Sunday, The long-term effects of President Trump’s tariffs won’t be known for a long time: remaking America’s industrial base is a generational task. It will even be months before the final tariff rates are agreed so a great deal of uncertainty remains. However, what is certain is that supply chains are being disrupted now but this is being overshadowed by the political noise and market watching. ...
Why we’re participating in the Metaculus Forecasting Tournament The Decis Intelligence agents weren’t designed to be forecasting tools. (In fact, I have a draft article explaining why we aren’t in the prediction business.) However, as I was building out the scenario planning tools, it was obvious that if these were to be useful, I needed a way to determine how effective, realistic, and accurate the scenario likelihood estimates were. But, as Andrew Ng wrote recently, evals and benchmarking are hard and expensive to set up. And in this case, how do you even evaluate the effectiveness of a forward-looking forecast? ...
Decis + HumanRisks We’re excited to announce our new partnership with HumanRisks (humanrisks.com). We’re integrating the Decis Country Intelligence feed directly into their comprehensive security risk management platform as a fully embedded data feed. This integration will provide security managers and asset owners with immediate access to Decis country risk ratings, directional assessments, and key news summaries—essential contextual information for making informed security decisions. The integration goes live on April 1st and will be available to both existing and new HumanRisks users on a trial basis. ...
Decis Audio Briefing Demo[ The world doesn’t need a new podcast (but you might) Andrew Sheves 🙀 My mind is officially blown. I tested Google’s Notebook LM (no, I don’t know why this took me so long to test) and am really impressed with the results. I gave it all of last week’s regional reports and it turned these into a conversation-style podcast. But it’s not just two voices reading the reports verbatim: it’s a natural-sounding discussion. ...
DeepSeek R1 vs Claude Sonnet 3.5 There’s been a lot of excitement around the Chinese Deepseek r1 model that was released last week. Deepseek is a reasoning, chain of thought (CoT) model that’s shaken things up due to the comparatively tiny budget ($ 6 million, which is probably OpenAI’s o1 cost to run its warm-up sequence) needed to train a frontier model of this sort. (There’s also been a fair amount of dread as US AI models are no longer the only game in town and Deepseek was considerably cheaper than an of the US alternatives. US firms were hammered in Monday’s trading with over $100bn ...
This is a ‘living’ article which will be updated as progress is made in tests and other research emerges. The original articile was written on Jaunuary 13 2025 and major edits will be dated. There are also demos of this process in action, the latest being the presentation to the Virtual AI Engineer Summit in NY in March 2025. Updates to add Utilize domain-specific heuristics (rules of thumb) for few-shot training ...
New Metrics Demo[ No Live Ratings Today Andrew Sheves Good morning - I’m transitioning the newsletters to a new system so am resynching everything, meaning that there are no updated country baseline ratings today. (These are still available and cuup to date in the workspace). Instead, here’s what the updated SITREP metrics will look like. Key Metrics WTI is $71.03 (Down) Low |—- < —————| High Wheat is $197.372 (Down) ...
I’ve had a lot of conversations about risk assessments recently and just finished a new feature in the Decis platform, both of which prompted this thought: What if we have things backwards? What if our risk assessment updated us instead of the other way around? We spend days — sometimes weeks — building a comprehensive risk assessment to understand our risks and build mitigation measures. But these are snapshots in time and can quickly become overtaken by events. That’s when we move from risk management to incident management or at minimum, we have to review our mitigation. ...
How to Use AI as a Security Risk Manager[ An AI AI White Paper Andrew Sheves (I’ll keep this short as I don’t want to suddenly spam your inbox after weeks of silence.) I’ve been collaborating with Doug Gray from HumanRisks on an AI white paper these last few months, and I’m pleased to announce that it’s now published. We took a much deeper dive into the topic than in any of our previous work, conducting a thorough review of how AI can assist security risk managers. We based this on the most recent academic studies of how knowledge workers make best use of these tools and aligned these AI benefits with identified gaps and challenges in Security Risk Management. ...
“Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tends to be the difficult ones.” Defense Secretary Donald Rumsfeld February 12, 2002. ...