• Spark Intelligence
  • Posts
  • Spark Intelligence #43: The AI literacy benchmark just landed – and you'll want to check yourself against it immediately

Spark Intelligence #43: The AI literacy benchmark just landed – and you'll want to check yourself against it immediately

The AI brief for creative leaders to grow your business and career, by Spark AI

👋 Greetings earthlings,

Emma here, co-founder of Spark AI. Let’s get straight into it - have you noticed this too: Most people think they're using AI pretty well. AND most people also feel like they're behind. Both things are true at once! It’s creating a kind of paralysis where nobody quite knows where they stand, making it’s easy to do nothing and take the ‘wait and see’ approach to taking any action.

The most common pattern in AI use that we see is what we call the vending machine mindset – type a vague prompt, read the first answer and decide whether it’s any good or not, and move on. It's not that people are using the wrong tools. It's that they're falling into treating AI like a search engine rather than a collaborator. This week we've got something that helps with that:

How good are you really at using AI?

Anthropic published research a few days ago that's the first thing I've seen from a major tech company that's very useful for measuring how ‘good’ we are at using AI. Anthropic’s AI Fluency Index analysed 9,830 conversations to identify the behaviours that actually distinguish effective AI users from everyone else.

Three findings stood out:

  1. The first is about iteration. People who go beyond the first answer and stay in the conversation are 5.6 times more likely to question the AI's reasoning, and 4 times more likely to spot what's missing. The single biggest predictor of how well someone uses AI is whether they treat it as a conversation or a search engine.

  2. The second is about setting expectations. Only 30% of people tell the AI how they want it to behave – things like "push back if my assumptions are wrong" or "walk me through your reasoning." That habit changes the quality of everything that follows.

  3. Third, when people use AI to actually build something – a document, a deck, a piece of content – they become more directive at the start (more likely to clarify the goal, specify the format, provide examples). But they become significantly less evaluative at the end. Less likely to identify what's missing, and less likely to question the reasoning. The more polished the output looks, the less people scrutinise it.

It's a useful contrast to what Accenture did last month, which was to tie senior promotions to weekly AI login frequency. An easy metric to measure, and the objective of what Accenture is trying to achieve is good - but login frequency isn't AI fluency. It's AI presence. Opening a tool every Tuesday proves nothing about whether someone is using it well. So what is AI Fluency? Read on.

Score yourself: The AI fluency check

Below are the 11 observable behaviours from the 4D AI Fluency Framework, developed by Professors Rick Dakan and Joseph Feller and used as the basis for Anthropic's research.

Be honest – tick the ones you genuinely do, most of the time.

Description – how you direct the AI

□ I iterate and refine rather than accepting the first response

□ I provide examples of what good looks like

□ I specify the format and structure I need

□ I tell the AI how I want it to interact with me – e.g. "push back if my assumptions are wrong"

□ I communicate the tone and style I'm after

□ I define who the output is for

Delegation – how you set up the task

□ I clarify my goal before asking AI to produce anything

□ I consult AI on the best approach before diving into execution

Discernment – how you evaluate what comes back

□ I identify when AI might be missing context

□ I question the reasoning when something doesn't hold up

□ I fact-check claims that matter, especially when the output looks polished

How did you do?

9–11: You're working the way the research says high-fluency users work. Now find out if your team is too.

5–8: Solid foundations. The gaps are probably in evaluation – the hardest habit to build when outputs look convincing.

Under 5: Vending machine territory. You're getting answers but leaving a lot of value on the table. The good news: iteration alone closes most of the gap.

I'm curious where we all actually sit on this. There are more than 2,000 agency and brand leaders reading this newsletter – so let's find out what the industry looks like in aggregate. Take two seconds to tell me your score, and I'll report back next week with the results.

The Anthropic framework maps almost identically to the core methodologies we teach across Spark's training programmes. That's not a coincidence – it's just what good AI practice looks like when you work it out from first principles. If you're trying to make the case internally for structured AI training rather than just letting people get on with it, this research is useful to have in your back pocket.

There's a related issue we see across agencies that the research doesn't capture. We estimate around half of AI activity is currently informal – people building their own workflows in isolation, without sharing them. When someone develops a brilliant custom approach and keeps it to themselves, two things happen: the rest of the team doesn't benefit, and that knowledge walks out the door when they leave.

Individual fluency is a start. Team fluency is where the commercial value sits.

One thing to try this week

A good way to use this with your team: send it out before your next team meeting and ask everyone to score themselves privately. Then, rather than asking people to share their scores, ask –

Which single behaviour on the list would make the biggest difference to your work if you did it consistently?

That conversation is usually much more useful than the number.

Our take on Nano Banana 2

Since its debut last August, Google’s Nano Banana model has become the centrepiece of the modern creative workflow. Last week Google dropped Nano Banana 2 - so I asked my co-founder Jules to share the inside scoop.

The end of the "reroll" era

Traditional diffusion models (think Midjourney, Flux or Ideogram) were brilliant at creating beautiful images but notoriously difficult to "art direct." If you liked a shot but needed to change a single detail, you were forced to rewrite the entire prompt and generate a whole new image - often altering the original composition, lighting, and character likeness in the process. While great for mood boarding, this lack of granular control made them a nightmare for prototyping and production.

The game changed when the leading LLMs (such as ChatGPT and Gemini) became truly multimodal. At first OpenAI led th way with GOT Image 1 but Google followed quickly with Nano Banana (official name Gemini 2.5 Flash Image - you can see why Nano Banana caught on!). With these multimodal models we moved beyond simple text-to-image. Because the model understands image and text simultaneously, they allow for non-destructive editing via natural language. Want to change a character’s sweater to red or swap a glass bottle for a sleek aluminium can? You just ask "Change the bottle to a can. Keep everything else the same". And it does it - the rest of the scene stays locked.

What’s new in Nano Banana 2?

Google’s naming conventions it's all over the place. We started with Nano Banana 2.5. Then back in December, we got Nano Banana Pro and now we have Nano Banana 2. This latest iteration focuses on three key improvements:

  • Better world knowledge - its integration with Gemini means it understands real world concepts and can create realistic infographics, specific places and more. 

  • Can now handle five consistent characters and fourteen consistent objects across generations, with more accurate text rendering, too. 

  • Richer details support generations of up to 4K 

The new standard

AI image generation is no longer just a shortcut for ideation. When integrated into dedicated workflows built in tools such as Weavy or Flora, Nano Banana Pro 2 provides the level of intentionality required for final production. If your agency still treats AI as a slightly random image generator rather than a precision production tool, you're missing out.

Our AI for creatives programme teaches people detailed image and video prompting to get consistency over their generations, and then helps them master creation and production at scale within Weavy. 

Next week (unless any mega AI news happens in the meantime - and you never know - it might!): why the middle management conversation matters more than the junior roles one, and what the Monks subscription model tells us about where commercial models are heading.

See you next time,

Co-founder, Spark AI

What did you think of our email today?

Login or Subscribe to participate in polls.

About Spark AI

Spark AI helps you lead your team through the biggest shift since digital, with AI training, transformation and tools. We've worked with 60+ agencies, published the only book on AI for Agencies, and teach at Oxford University.

👉 Find out more 

Not a subscriber yet?