Dottor Anghel AI

⚠️ 𝗖𝗢𝗡𝗧𝗘𝗫𝗧 𝗥𝗢𝗧 𝗜𝗦𝗡'𝗧 𝗔 𝗕𝗨𝗚. 𝗜𝗧'𝗦 𝗔 𝗙𝗨𝗡𝗗𝗔𝗠𝗘𝗡𝗧𝗔𝗟 𝗥𝗜𝗚𝗛𝗧𝗦 𝗥𝗜𝗦𝗞.

If you read my previous post on Ralph Loop, you know LLM performance degrades as context grows. Scientifically documented. Unavoidable.

But here's what nobody is talking about: context rot isn't a "performance issue." It's a systemic risk that should be in every FRIA under the AI Act.

And I've never seen it there.

𝗔 𝗥𝗘𝗔𝗟 𝗦𝗖𝗘𝗡𝗔𝗥𝗜𝗢

An HR platform uses AI to screen 500+ CVs daily.

9:00 AM - Candidate #1 (Mario):
→ AI session starts fresh
→ Context: 500 tokens
→ Performance: optimal
→ Score: 85/100 → hired

5:00 PM - Candidate #300 (Sara):
→ Same AI session (8 hours running)
→ Context: 45,000 tokens (299 CVs processed before her)
→ Performance: degraded by context rot
→ Score: 72/100 → rejected

Sara's CV was identical to Mario's. Same skills, same experience, same job posting.

She was rejected because the AI processed her last, not because she was less qualified.

That's not a bug. That's systematic discrimination based on processing order.

𝗪𝗛𝗬 𝗧𝗛𝗜𝗦 𝗜𝗦 𝗔 𝗙𝗥𝗜𝗔 𝗣𝗥𝗢𝗕𝗟𝗘𝗠

FRIA requires identifying risks to non-discrimination and fair treatment.

"Continuous State" architectures (like Ralph Loop's second implementation) create this risk by design:
❌ Performance varies by position in sequence
❌ Early inputs get better treatment than late inputs
❌ Order-based discrimination is architecturally guaranteed

"Clean State" architectures mitigate it:
✓ Context resets for each input
✓ Consistent performance across all candidates
✓ No order-based bias

One architecture creates fundamental rights risk. The other prevents it.

𝗪𝗛𝗔𝗧 𝗧𝗛𝗜𝗦 𝗠𝗘𝗔𝗡𝗦 𝗙𝗢𝗥 𝗖𝗢𝗠𝗣𝗟𝗜𝗔𝗡𝗖𝗘

If you're conducting FRIA for AI systems:

𝗤𝗨𝗘𝗦𝗧𝗜𝗢𝗡 𝟭: Does your system accumulate context over time?
If yes → you have documented performance degradation risk.

𝗤𝗨𝗘𝗦𝗧𝗜𝗢𝗡 𝟮: Does processing order affect outcomes?
If "Continuous State" → the answer is mathematically yes.

𝗤𝗨𝗘𝗦𝗧𝗜𝗢𝗡 𝟯: Have you documented context rot as a fundamental rights risk?
If no → your FRIA is incomplete.

𝗧𝗛𝗘 𝗟𝗜𝗔𝗕𝗜𝗟𝗜𝗧𝗬 𝗤𝗨𝗘𝗦𝗧𝗜𝗢𝗡

When Sara files a GDPR complaint showing identical CVs but different outcomes, who's liable?

→ The vendor, for choosing vulnerable architecture?
→ The deployer, for not identifying this systemic risk?
→ Both?

The AI Act doesn't answer this yet. But auditors will ask.

𝗛𝗢𝗪 𝗜 𝗔𝗣𝗣𝗥𝗢𝗔𝗖𝗛 𝗧𝗛𝗜𝗦

I don't conduct "generic" FRIAs based on use case alone.

I assess architectural risks:
✓ Does the architecture create systematic performance variance?
✓ Can processing order introduce discrimination?
✓ Is context rot documented and mitigated?
✓ Are vendor contracts clear on liability for architecture-induced bias?

Because an HR AI that discriminates based on when you applied (not what you submitted) violates fundamental rights.

Even if nobody intended it. Even if "it just happened" because of context rot.

Intent doesn't matter. Impact does.

📩 Conducting FRIA for AI systems with autonomous agents? dott.anghel.ai@gmail.com

#AIAct #FRIA #FundamentalRights #ContextRot #AIGovernance #Compliance #Discrimination #TechLaw #RalphLoop #AIAgents

7 months ago | [YT] | 3

Dottor Anghel AI

🎭 𝗥𝗔𝗟𝗣𝗛 𝗪𝗜𝗚𝗚𝗨𝗠 𝗔𝗡𝗗 𝗧𝗛𝗘 𝗔𝗜 𝗔𝗚𝗘𝗡𝗧 𝗧𝗛𝗔𝗧 𝗪𝗜𝗡𝗦 𝗕𝗬 𝗡𝗘𝗩𝗘𝗥 𝗤𝗨𝗜𝗧𝗧𝗜𝗡𝗚

In 2024, Geoffrey Huntley built an AI coding agent named after the least competent character from The Simpsons: Ralph Wiggum, the kid who said "I'm learnding!" and ate paste.

The joke was perfect. Ralph doesn't win by being smart. It wins by being persistent.

𝗧𝗛𝗘 𝗢𝗥𝗜𝗚𝗜𝗡𝗔𝗟: 𝗢𝗡𝗘 𝗟𝗜𝗡𝗘 𝗢𝗙 𝗕𝗔𝗦𝗛

while :; do cat PROMPT.md | claude-code ; done

Loop forever. Feed the prompt to Claude. If it fails, try again.

The philosophy: eventual consistency. Try enough times, the AI will produce working code.

It built complete projects, a new programming language, and six production repos in one night during a Y Combinator hackathon.

𝗧𝗛𝗘 𝗘𝗩𝗢𝗟𝗨𝗧𝗜𝗢𝗡: 𝗧𝗪𝗢 𝗣𝗛𝗜𝗟𝗢𝗦𝗢𝗣𝗛𝗜𝗘𝗦

As Ralph gained traction, two camps emerged:

𝗖𝗟𝗘𝗔𝗡 𝗦𝗧𝗔𝗧𝗘 (snarktank/ralph):
Kill the session after every task. Start fresh.
→ New Claude instance each iteration
→ Context always minimal
→ State persists externally: git, progress.txt, prd.json

𝗖𝗢𝗡𝗧𝗜𝗡𝗨𝗢𝗨𝗦 𝗦𝗧𝗔𝗧𝗘 (Claude Code plugin):
Keep the session alive. Loop within one conversation.
→ One long session, never terminates
→ Context accumulates indefinitely
→ State persists in model's memory
→ Requires circuit breakers, timeouts, limits

Same goal. Opposite execution.

𝗧𝗛𝗘 𝗣𝗥𝗢𝗕𝗟𝗘𝗠: 𝗖𝗢𝗡𝗧𝗘𝗫𝗧 𝗥𝗢𝗧

LLMs don't process token 10,000 like token 100. Performance degrades as context grows. Always.

Chroma's research documented "context rot":
→ Longer context = worse performance
→ Past information becomes distractors
→ Hallucination rates increase
→ Models get less reliable over time

LongMemEval proved it: focused input outperforms full context, even when full context contains more information.

𝗪𝗛𝗬 𝗜𝗧 𝗠𝗔𝗧𝗧𝗘𝗥𝗦

"Clean State" is architecturally aligned with context rot. It fights the problem by design.

"Continuous State" is architecturally vulnerable. It needs complex scaffolding to prevent performance collapse.

One works with the science. The other against it.

𝗧𝗛𝗘 𝗚𝗢𝗩𝗘𝗥𝗡𝗔𝗡𝗖𝗘 𝗤𝗨𝗘𝗦𝗧𝗜𝗢𝗡

If you're deploying autonomous AI agents:

→ Can you document which architecture your system uses?
→ Have you assessed context rot as a risk factor?
→ Can you prove reliability doesn't degrade over time?

When a regulator asks "how does your AI maintain consistent performance?", "it just works" isn't documentation.

Understanding whether your system works with or against fundamental LLM limitations is.

Ralph Wiggum succeeded because it embraced persistence over perfection. Your AI governance should do the same: build systems that work with the technology's limits, not against them.

📩 Building or procuring autonomous AI systems? dott.anghel.ai@gmail.com

#AI #AIGovernance #RalphLoop #LLM #ContextRot #AIAgents #Compliance #MachineLearning #AIAct #TechLaw

7 months ago | [YT] | 3

Dottor Anghel AI

🎨 𝗛𝗢𝗪 𝗔𝗜 "𝗜𝗠𝗔𝗚𝗜𝗡𝗘𝗦" 𝗪𝗛𝗔𝗧 𝗗𝗢𝗘𝗦𝗡'𝗧 𝗘𝗫𝗜𝗦𝗧: 𝗧𝗪𝗢 𝗠𝗢𝗗𝗘𝗟𝗦 𝗖𝗢𝗠𝗣𝗔𝗥𝗘𝗗

When you ask Midjourney or DALL-E to generate an image, what really happens under the hood?

The answer isn't just one. There are two completely different philosophies for "creating from nothing":

𝟭. 𝗗𝗜𝗙𝗙𝗨𝗦𝗜𝗢𝗡 𝗠𝗢𝗗𝗘𝗟𝗦 (the "clearing noise" method)
How it works: start from pure random noise and gradually "clean it up" until you get the desired image
✅ Pros: exceptional photographic quality, fine detail control
❌ Cons: computationally expensive, requires many iterative steps
📌 Examples: Stable Diffusion, DALL-E, Midjourney

𝟮. 𝗔𝗨𝗧𝗢𝗥𝗘𝗚𝗥𝗘𝗦𝗦𝗜𝗩𝗘 𝗠𝗢𝗗𝗘𝗟𝗦 (the "pixel by pixel" method)
How it works: generates the image one token at a time, just like ChatGPT writes text word by word
✅ Pros: unified text+image architecture, scalability, speed in certain contexts
❌ Cons: historically inferior quality for realistic images
📌 Examples: DALL-E 3 (hybrid), visual transformer models

𝗕𝗨𝗧 𝗧𝗛𝗘𝗥𝗘'𝗦 𝗔 𝗡𝗘𝗪 𝗜𝗗𝗘𝗔 𝗧𝗛𝗔𝗧 𝗖𝗛𝗔𝗡𝗚𝗘𝗦 𝗘𝗩𝗘𝗥𝗬𝗧𝗛𝗜𝗡𝗚

Researchers are developing hybrid models like 𝗚𝗟𝗠-𝗜𝗺𝗮𝗴𝗲 that combine the best of both worlds:
→ The efficiency of autoregressive
→ The quality of diffusion
→ A single architecture for text, image, video

In the video, I show you with clear animations:
✓ How diffusion "removes noise" step by step
✓ How autoregressive "builds" the image token by token
✓ Why GLM-Image could be the future of multimodal generation

𝗪𝗛𝗬 𝗦𝗛𝗢𝗨𝗟𝗗 𝗬𝗢𝗨 𝗞𝗡𝗢𝗪 𝗧𝗛𝗘𝗦𝗘 𝗗𝗜𝗙𝗙𝗘𝗥𝗘𝗡𝗖𝗘𝗦?

If you work with generative AI (or are considering implementing it), knowing which model your tool uses changes:
• The performance you can expect
• The actual computational costs
• The technical (and legal) limits of generation
• How to document the process for AI Act compliance

Understanding the architecture isn't just technical curiosity: it's conscious governance.

📽️ Watch the full video to see the differences in action (6 min) -> https://www.youtube.com/watch?v=B-pYz...

#AI #GenerativeAI #ImageGeneration #AIAct #Diffusion #Transformer #TechEducation #MachineLearning #AIGovernance #DeepLearning

7 months ago | [YT] | 6

Dottor Anghel AI

♀️ 𝗔𝗜 𝗔𝗡𝗗 𝗩𝗜𝗢𝗟𝗘𝗡𝗖𝗘 𝗔𝗚𝗔𝗜𝗡𝗦𝗧 𝗪𝗢𝗠𝗘𝗡: 𝗠𝗬 𝗜𝗡𝗧𝗘𝗥𝗩𝗘𝗡𝗧𝗜𝗢𝗡 𝗔𝗧 𝗧𝗛𝗘 𝗖𝗛𝗔𝗠𝗕𝗘𝗥 𝗢𝗙 𝗗𝗘𝗣𝗨𝗧𝗜𝗘𝗦

📍 Yesterday, at the Chamber of Deputies, I had the honour of addressing one of the most urgent and troubling challenges of our time.

The new frontier of violence against women increasingly takes the form of digital abuse, identity manipulation and reputational harm enabled by artificial intelligence and deepfake technologies.

As a jurist and expert in artificial intelligence and emerging technologies, I delivered an intervention entitled “AI, Deepfakes and the New Frontier of Violence Against Women”, focusing on:

🇪🇺 The European regulatory framework (AI Act, Directive (EU) 2024/1385, Digital Services Act)

🇮🇹 The Italian legal response, with Law no. 132/2025 and the introduction of Article 612-quater of the Italian Criminal Code, criminalising the dissemination of deepfake content.

𝗜𝗡𝗦𝗧𝗜𝗧𝗨𝗧𝗜𝗢𝗡𝗔𝗟 𝗔𝗖𝗞𝗡𝗢𝗪𝗟𝗘𝗗𝗚𝗘𝗠𝗘𝗡𝗧𝗦

I wish to express my sincere gratitude to:

🇷🇴 H.E. Gabriela Dancau, Ambassador of Romania to Italy, San Marino and Malta,

🇸🇲 H.E. Marina Emiliani, Ambassador of the Republic of San Marino to Romania,

for their presence and sensitivity towards a phenomenon that deeply affects women’s dignity, honour and identity.

My thanks also go to Hon. Luciano Ciocchetti, organiser of the initiative, and to Hon. Martina Semenzato, President of the Parliamentary Commission of Inquiry on Femicide and all forms of gender-based violence, for making this institutional discussion possible.

𝗕𝗘𝗬𝗢𝗡𝗗 𝗟𝗘𝗚𝗜𝗦𝗟𝗔𝗧𝗜𝗢𝗡

I would also like to thank Gianluca Mech, creator of the short film “La Trappola di Venere”, for his contribution to a cultural and preventive intervention addressed to men, aimed at recognising psychological and emotional warning signs before distress turns into violence.

The myth of Venus serves as a metaphor for the loss of clarity caused by obsessive love.

Events like this show that effective protection requires action on multiple levels: legal, technological, educational and cultural.

#AI #Deepfakes #ViolenceAgainstWomen #AIAct #DigitalLaw #AIGovernance #GenderBasedViolence

Skip the Carousel

1 2 3 4 5 6 7 8 9 10

7 months ago | [YT] | 7

Dottor Anghel AI

📌 𝗥𝗘𝗣𝗘𝗧𝗜𝗧𝗔 𝗜𝗨𝗩𝗔𝗡𝗧: 𝗪𝗛𝗬 𝗗𝗨𝗣𝗟𝗜𝗖𝗔𝗧𝗜𝗡𝗚 𝗬𝗢𝗨𝗥 𝗣𝗥𝗢𝗠𝗣𝗧 𝗠𝗜𝗚𝗛𝗧 𝗕𝗘 𝗧𝗛𝗘 𝗦𝗜𝗠𝗣𝗟𝗘𝗦𝗧 𝗪𝗔𝗬 𝗧𝗢 𝗕𝗢𝗢𝗦𝗧 𝗟𝗟𝗠 𝗣𝗘𝗥𝗙𝗢𝗥𝗠𝗔𝗡𝗖𝗘

Google Research just published a study confirming what your grandmother always told you: repetition works.

And it's almost embarrassingly simple.

Take your prompt. Copy-paste it twice. That's it. No complex Chain-of-Thought. No advanced engineering. Just: Prompt + Prompt = Better Results.

𝗪𝗛𝗬 𝗗𝗢𝗘𝗦 𝗧𝗛𝗜𝗦 𝗪𝗢𝗥𝗞?

LLMs are causal models: they process tokens left-to-right and can't "look ahead".

✅ Single prompt → the model processes each token as it arrives, with limited context.

✅ Duplicated prompt → when the model reaches the second repetition, it can attend to the full prompt from the first pass.

It's like saying: "Read this carefully, form a complete picture, and NOW answer me."

𝗧𝗛𝗘 𝗥𝗘𝗦𝗨𝗟𝗧𝗦 𝗔𝗥𝗘 𝗦𝗧𝗔𝗚𝗚𝗘𝗥𝗜𝗡𝗚

Google tested this across 70 benchmarks (ARC, GSM8K, MMLU-Pro, MATH, etc.):

✅
- 47 wins out of 70 tests
- 0 losses (yes, zero)
- Accuracy jumps from 21.33% to 97.33% in some tasks
- Works across Gemini, GPT-4o, Claude 3.5, DeepSeek
- No latency increase: pre-fill phase is parallel, so response time stays the same

❌
- Only applies to non-reasoning models (standard inference, not o1-style deliberation)
- Effect varies by task and model architecture

𝗪𝗛𝗔𝗧 𝗧𝗛𝗜𝗦 𝗠𝗘𝗔𝗡𝗦 𝗙𝗢𝗥 𝗣𝗥𝗢𝗗𝗨𝗖𝗧𝗜𝗢𝗡 𝗦𝗬𝗦𝗧𝗘𝗠𝗦

For teams running production LLMs, this is a zero-cost performance upgrade:

1. Drop-in implementation: modify your prompt wrapper to duplicate input
2. No infrastructure changes: same API, same latency
3. Immediate gains on structured tasks: reasoning, Q&A, classification

But here's the governance angle most teams miss:

If you're documenting AI system behavior for compliance (AI Act Art. 13, transparency requirements), prompt repetition gives you a reproducible, explainable intervention you can defend to auditors.

You're not changing the model. You're not injecting opaque vectors. You're just… giving it more time to think. Auditors love simplicity.

𝗪𝗛𝗘𝗡 𝗧𝗢 𝗨𝗦𝗘 𝗜𝗧 (𝗔𝗡𝗗 𝗪𝗛𝗘𝗡 𝗡𝗢𝗧 𝗧𝗢)

✅ USE IT FOR:
- Structured reasoning tasks (math, logic, multi-step questions)
- High-stakes outputs where accuracy > cost
- Non-reasoning models (GPT-4, Claude 3.5, Gemini 2.0)

❌ DON'T USE IT FOR:
- Reasoning models (o1, DeepSeek-R1) → they already do internal iteration
- Creative/open-ended generation → repetition doesn't help much
- Latency-critical applications where token count matters

Sometimes we overcomplicate things. We chase RAG pipelines, fine-tuning, complex prompt chains.

And the solution is literally: say it twice.

📄 Link to the full paper in the first comment

📩 If you want to integrate simple, auditable prompt strategies into your AI compliance stack, contact me at: dott.anghel.ai@gmail.com

#AI #PromptEngineering #LLM #AIAct #Compliance #AIGovernance #GoogleResearch #FRIA

7 months ago | [YT] | 7

Dottor Anghel AI

📌 𝗡𝗘𝗨𝗥𝗢𝗡𝗣𝗘𝗗𝗜𝗔: 𝗧𝗛𝗘 “𝗪𝗜𝗞𝗜𝗣𝗘𝗗𝗜𝗔 𝗢𝗙 𝗡𝗘𝗨𝗥𝗢𝗡𝗦” 𝗙𝗢𝗥 𝗟𝗟𝗠𝗦

In modern LLMs we talk about “neurons” and “features”, but most teams never see what they actually do.
Neuronpedia tries to fix that by turning neuron/SAE interpretability into a shared, navigable resource.

1️⃣ 𝗪𝗛𝗔𝗧 𝗡𝗘𝗨𝗥𝗢𝗡𝗣𝗘𝗗𝗜𝗔 𝗜𝗦

A public, collaborative atlas of neurons and SAE features for real models.

PRO ✅
- Explanations, example prompts, and activations for individual neurons/features.
- Strong focus on sparse autoencoders (SAEs) and their interpretable “features”.
- Web UI + APIs so you can browse, tag, and analyse features without building your own tooling.

2️⃣ 𝗪𝗛𝗔𝗧 𝗬𝗢𝗨 𝗖𝗔𝗡 𝗗𝗢 𝗪𝗜𝗧𝗛 𝗜𝗧

Think of Neuronpedia as your starting point for mech‑interp and steering.

PRO ✅
- Inspect concrete features like “legal tone”, “Eiffel Tower”, or “self‑harm” with real examples.
- Compare features across layers/models and plug SAEs into existing analysis libraries.

CON ❌ (𝗶𝗳 𝘆𝗼𝘂 𝗶𝗴𝗻𝗼𝗿𝗲 𝗶𝘁)
- You keep treating steering vectors as black‑box magic instead of grounded, documented concepts.

3️⃣ 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚 & 𝗦𝗔𝗙𝗘𝗧𝗬 𝗔𝗡𝗚𝗟𝗘

Neuronpedia is also a bridge between interpretability and real‑world control.

- Use SAE features as steering knobs: up‑ or down‑weight a feature and see how outputs change.
- Design safety/style steering grounded in labeled features, not opaque directions, which reduces side‑effects and helps with governance.

If you care about serious governance, this is a big upgrade over “we changed the system prompt and it looks better”.

4️⃣ 𝗪𝗛𝗬 𝗣𝗥𝗔𝗖𝗧𝗜𝗧𝗜𝗢𝗡𝗘𝗥𝗦 𝗦𝗛𝗢𝗨𝗟𝗗 𝗖𝗔𝗥𝗘

- Cuts time‑to‑experiment: hosting, visualisation, and collaboration are handled for you.
- Makes interventions easier to justify to stakeholders and regulators: you can point to specific, named features with examples, not just “prompt engineering that seems to work”.

📩 If you want to explore how Neuronpedia and steering vectors can fit into your AI governance or compliance stack, contact me at: dott.anghel.ai@gmail.com

#AI #Compliance #FRIA #AIAct #Neuronpedia

8 months ago | [YT] | 7

Dottor Anghel AI

📌 𝗠𝗢𝗥𝗘 𝗙𝗘𝗔𝗧𝗨𝗥𝗘𝗦, 𝗠𝗢𝗥𝗘 𝗖𝗢𝗡𝗧𝗥𝗢𝗟? 𝗡𝗢𝗧 𝗥𝗘𝗔𝗟𝗟𝗬. 𝗦𝗜𝗡𝗚𝗟𝗘‑𝗙𝗘𝗔𝗧𝗨𝗥𝗘 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚 𝗢𝗙𝗧𝗘𝗡 𝗪𝗜𝗡𝗦

In the SAE/steering world it sounds intuitive: “more features = more control”.

Recent work suggests something else: more poorly chosen features can actually reduce coherence and control.

1️⃣ 𝗦𝗜𝗡𝗚𝗟𝗘‑𝗙𝗘𝗔𝗧𝗨𝗥𝗘 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚: 𝗦𝗡𝗜𝗣𝗘𝗥 𝗠𝗢𝗗𝗘

With an SAE, a single good feature is often enough to steer a behavior: refusal, legal tone, marketing style, etc.

✅

1. Clear, interpretable effect

2. Less interference with other behaviors

3. Easier to test, document, and defend in audits

❌ (when you mix too many features)

1. Some mostly “read” the input instead of driving the output

2. Vectors add up and introduce noise → text becomes less coherent, control less predictable

Moral: one well‑chosen feature > many “vibes‑based” features.

2️⃣ 𝗠𝗨𝗟𝗧𝗜‑𝗟𝗘𝗩𝗘𝗟 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚: 𝗣𝗢𝗧𝗘𝗡𝗧𝗜𝗔𝗟, 𝗕𝗨𝗧 𝗡𝗢𝗧 𝗠𝗔𝗚𝗜𝗖

Intervening on multiple layers / intermediate levels only helps if you know what you are doing.

✅

1. Some layers are better for content (middle), others for style/output (late)

2. Picking 1–3 “key” layers often beats “steer everywhere”

❌

There is no strong evidence that “applying the same feature on many layers” systematically beats using it where it has the most impact:

you risk more complexity, more noise, and governance explanations nobody really believes.

3️⃣ 𝗧𝗛𝗘 𝗥𝗘𝗔𝗟 𝗧𝗥𝗜𝗖𝗞: 𝗥𝗜𝗚𝗛𝗧 𝗩𝗘𝗖𝗧𝗢𝗥 + 𝗪𝗘𝗟𝗟‑𝗧𝗨𝗡𝗘𝗗 𝗗𝗘𝗖𝗢𝗗𝗜𝗡𝗚

In practical tests (including Eiffel‑Tower‑style demos), the quality jump does not come from “more features” but from:

1. Clamping

Limit how much the steering vector can distort activations → the concept stays, with less overshooting and fewer weird outputs.

2. Lower temperature

Reduce sampling temperature → less randomness, better instruction following, the steering signal is not washed out.

3. Light repetition penalty

Apply a moderate repetition penalty → the concept remains present without obsessive repetition.

Net result: compared to “simple addition + standard sampling”, the combo single well‑chosen feature + clamping + lower T + light repetition penalty often yields outputs that are more coherent, better aligned, and still strongly express the target concept.


★ BONUS → The same steering coefficient is not universal: its effect can swing a lot with different prompts and contexts. Recent work shows that for a fixed coefficient some inputs get strong steering, others weak or even inverted effects, so you must treat it as a prompt‑ and task‑dependent hyperparameter.



---

If you care about serious governance, the question is not “how many features can I turn on?”, but:

“Which few features can I actually explain, control, and document – and how do I set up decoding around them?”

#AIAct #Compliance #Steering #Feature #AI #LegalTech

8 months ago | [YT] | 7

Dottor Anghel AI

📌 𝗣𝗥𝗢𝗠𝗣𝗧 𝗘𝗡𝗚𝗜𝗡𝗘𝗘𝗥𝗜𝗡𝗚 𝗩𝗦 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚: 𝗪𝗛𝗢 𝗦𝗛𝗢𝗨𝗟𝗗 𝗖‑𝗟𝗘𝗩𝗘𝗟𝗦 𝗕𝗘𝗧 𝗢𝗡?

Many companies are hiring prompt engineers.
Very few are building steering infrastructure.

If you need to govern models in production, this isn’t a “style” choice – it’s a choice about control, risk, and scalability.

1️⃣ PROMPT ENGINEERING

✅ PROS
1. No model access needed: the API is enough.
2. Fast iteration: you can tweak prompts in minutes.
3. Non‑technical teams can contribute.

❌ CONS
1. Fragile: small changes in context can break behavior.
2. Hard to scale: each team “invents” its own prompts, so cross‑product / cross‑country consistency is weak.
3. Auditing is painful: it’s hard to explain to an auditor why a specific output appeared.

2️⃣ STEERING

✅ PROS
1. Internal control: you act on hidden states with concept vectors (risk, tone, safety, etc.).
2. Continuous adjustment: X_next = X + α V → you dial behavior up or down instead of rewriting prompts.
3. Central governance: you can define “corporate” vector libraries and reuse them across products and use cases.

❌ CONS
1. Needs low‑level access (self‑hosted models or advanced tooling).
2. Technical setup: extracting, testing, and documenting robust vectors is not a side project.
3. Not every behavior is a “single direction”: some vectors are less reliable and must be refined.


3️⃣ HOW TO IMPROVE STEERING: FROM “JUST ADD” TO FINE‑GRAIN CONTROL

The basic scheme is:
- addition: X_next = X + α V
- standard sampling: default temperature, no extra tuning.

Practical tests show you can make steering much more stable with:

1. Clamping
Limit how much the vector can deform activations (per dimension, norm, or token).
Effect: the concept stays, but you avoid overshooting and weird outputs.

2. Lower temperature
Reduce sampling temperature.
Effect: less randomness, stronger adherence to instructions, the steering vector isn’t washed out by noise.

3. Light repetition penalty
Apply a moderate repetition penalty.
Effect: the target concept remains present without obsessive, repetitive outputs.

Net result: compared to “addition + standard sampling”, the combo clamping + lower T + light repetition penalty produces smoother responses, better instruction‑following, and equal or better preservation of the target concept.


For prototypes and low‑risk use cases, prompt engineering is enough.
For critical, auditable, multi‑team products, you need to move from “writing prompts” to designing real steering levers.

#AIAct #AI #PromptEngineering #Steering #Compliance

8 months ago | [YT] | 7

Dottor Anghel AI

🧩 𝗦𝗣𝗔𝗥𝗦𝗘 𝗔𝗨𝗧𝗢𝗘𝗡𝗖𝗢𝗗𝗘𝗥𝗦 (𝗦𝗔𝗘): 𝗧𝗛𝗘 𝗛𝗜𝗗𝗗𝗘𝗡 𝗠𝗔𝗣 𝗬𝗢𝗨 𝗦𝗛𝗢𝗨𝗟𝗗 𝗕𝗘 𝗨𝗦𝗜𝗡𝗚 𝗧𝗢 𝗚𝗢𝗩𝗘𝗥𝗡 𝗬𝗢𝗨𝗥 𝗟𝗟𝗠𝗦

Most companies control AI with prompts and policies.
SAEs let you control it with internal switches, instead of hoping the model “behaves”.

Steering tells you how to push a model.
Concept vectors tell you in which direction.
Sparse Autoencoders (SAE) tell you what is actually inside, in a readable way.


1️⃣ 𝗪𝗛𝗔𝗧 𝗔𝗡 𝗦𝗔𝗘 𝗗𝗢𝗘𝗦 (𝗦𝗜𝗠𝗣𝗟𝗜𝗙𝗜𝗘𝗗)

An SAE takes an LLM’s activations (hidden states) and recodes them into:
1. a larger LATENT space
2. that is SPARSE: almost all values are zero.

Forcing sparsity has a key effect:
each active feature tends to represent a cleaner, more interpretable concept (or a small cluster of nearby concepts).

Instead of “polysemantic” neurons doing a bit of everything, you get a list of switches:
1. “legalese”
2. “informal language”
3. “hate speech”
4. “code / snippets”
etc.

In practice, the SAE turns the chaos of activations into a switchboard, where each button turns on a specific behavior of the model.

And because there are thousands of features, you can use LLMs themselves to help name each switch (“this feature fires on phrases X: looks like ‘polite tone’, ‘legalese’, etc.”) – this is auto‑interpretability.

2️⃣ 𝗛𝗢𝗪 𝗦𝗔𝗘𝗦, 𝗖𝗢𝗡𝗖𝗘𝗣𝗧 𝗩𝗘𝗖𝗧𝗢𝗥𝗦 & 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚 𝗙𝗜𝗧 𝗧𝗢𝗚𝗘𝗧𝗛𝗘𝗥

Each sparse SAE feature has:
1. a vector in latent space (the “shape” of the concept)
2. an activation weight (how strongly it is on for that token / sequence).

This means you can:
1. Identify which features fire on certain concepts (toxicity, PII leaks, non‑compliant tone).
2. Treat that feature as a concept vector: the direction in latent space representing that behavior.

From there you go back to the steering formula:

X_next = X + V * α

- X = current internal state of the model
- V = vector of the SAE feature (the concept direction)
- α = how much you amplify or damp that concept

It’s the same mechanism as steering vectors: you press an internal button and push the model’s behavior in that direction.

3️⃣ 𝗪𝗛𝗬 𝗟𝗘𝗚𝗔𝗟 / 𝗖𝗧𝗢 / 𝗥𝗜𝗦𝗞 𝗦𝗛𝗢𝗨𝗟𝗗 𝗖𝗔𝗥𝗘

For AI governance, SAEs matter because they let you:
1. Map “risk zones” inside the model (features that light up on problematic content).
2. Build stable technical controls: not just prompts and policies, but mathematical levers over internal concepts.

It’s the shift from:
“We told the model not to do it”
to
“We directly lowered the activation of the features that cause that behavior”.

➡️ 𝗡𝗘𝗫𝗧 𝗣𝗢𝗦𝗧: 𝗣𝗥𝗢𝗠𝗣𝗧 𝗘𝗡𝗚𝗜𝗡𝗘𝗘𝗥𝗜𝗡𝗚 𝗩𝗦 𝗦𝗧𝗘𝗘𝗥𝗜𝗡𝗚
Which one actually gives you control at scale? We’ll compare costs, risks, and trade‑offs from a C‑level and control‑function perspective, not a “prompt hacker” one.

#AIAct #AI #SAE

8 months ago (edited) | [YT] | 7

Dottor Anghel AI

🧠 𝗖𝗢𝗡𝗖𝗘𝗣𝗧 𝗩𝗘𝗖𝗧𝗢𝗥𝗦: 𝗪𝗛𝗘𝗡 𝗔 𝗖𝗢𝗡𝗖𝗘𝗣𝗧 𝗕𝗘𝗖𝗢𝗠𝗘𝗦 𝗔 𝗠𝗔𝗧𝗛𝗘𝗠𝗔𝗧𝗜𝗖𝗔𝗟 𝗗𝗜𝗥𝗘𝗖𝗧𝗜𝗢𝗡

In previous posts we talked about:

- steering = changing an LLM’s behavior “on the fly”
- concept embeddings & activation space = the internal map you steer on

Now comes the key piece: CONCEPT VECTORS.

A concept vector is a DIRECTION in activation space that corresponds to a specific concept:

“legal tone”, “marketing style”, “toxicity”, “risk conservatism”.

📌 𝗙𝗿𝗼𝗺 𝗵𝗶𝗱𝗱𝗲𝗻 𝘀𝘁𝗮𝘁𝗲 𝘁𝗼 “𝘀𝘁𝗲𝗲𝗿𝗲𝗱” 𝘀𝘁𝗮𝘁𝗲

Imagine a hidden state vector of an LLM at some layer:

Hidden State Vector:

X = [1; 3; 6; 7]

Now suppose we’ve found a concept vector:

V = [−1; 0; 2; 1] (example)

When we do steering, we build a new state:

X_next = X + V * 𝜶

where α is a coefficient that controls how strongly we apply that concept (the “intensity knob”).

In practice:

- X = how the model was “thinking” before
- V = the direction “more legal”, “less toxic”, “more conservative”
- 𝜶 = the dial: 0.2, 0.5, 1.5… how hard you push the concept

Mathematically, it’s just vector addition.

Operationally, it’s a personality/behavior shift without touching the model’s weights.

⚙️ 𝗛𝗢𝗪 𝗪𝗘 𝗙𝗜𝗡𝗗 𝗧𝗛𝗘 𝗩𝗘𝗖𝗧𝗢𝗥 𝗩

Two main (simplified) paths:

1️⃣ Difference between two prompt groups

- Group A: prompts that EXPRESS the concept (e.g. toxic answers).
- Group B: prompts that DO NOT express it (e.g. neutral answers).

You compute the mean activations for A and for B, then:

V ≈ mean_activations(A) − mean_activations(B)

This V points in the direction “more like A, less like B”.

2️⃣ Sparse Autoencoders (SAE) – teaser

Sparse Autoencoders take the model’s activations and rewrite them into a larger but SPARSE space (mostly zeros).

Each sparse “feature” often corresponds to an internal concept (or near‑concept): an interpretable pattern of behavior.

In short:

- the SAE gives you a “button” (a feature) that turns a concept on/off
- that button corresponds to a vector in latent space you can use to steer the model

SAEs deserve their own post, so we’ll go deeper next time.

🧭 𝗪𝗛𝗬 𝗖𝗢𝗡𝗖𝗘𝗣𝗧 𝗩𝗘𝗖𝗧𝗢𝗥𝗦 𝗠𝗔𝗧𝗧𝗘𝗥 𝗙𝗢𝗥 𝗚𝗢𝗩𝗘𝗥𝗡𝗔𝗡𝗖𝗘

They let you turn:

- “this model is too aggressive/creative”

into

- “I apply +0.3 on prudence, −0.2 on creativity in its internal activations”.

It’s the shift from written policies to mathematical levers: controllable, measurable, reproducible.

➡️ 𝗡𝗘𝗫𝗧 𝗣𝗢𝗦𝗧: Sparse Autoencoders and how to use them to discover and control hidden concepts in your LLMs (without manual reverse‑engineering).

📩 Want help mapping concept vectors for risk, tone, or safety in your own AI systems – and turning them into real governance levers, not just prompts? Write to me confidentially: dott.anghel.ai@gmail.com

#AI #AIAct #FRIA #ConceptVectors

8 months ago (edited) | [YT] | 7