Five tiny experiments. For each one, guess what will happen first, then run it. No install,
nothing leaves this page. When these click, the real bugs are just these ideas in a real editor.
The one idea: computers constantly mix up two things, data (content to show or store) and
instructions (commands to obey). Almost every hack is a piece of data that sneaks into a place
where the computer treats it as an instruction.
The picture to hold: the assistant is an overly literal intern who does exactly what any note
says and cannot tell which notes came from you and which an attacker slipped onto its desk.
Experiment 1
Text can be an action
This little box obeys typed commands: red, green, time, or
add 2 3.
Type red and run it. What happens?
ready
Your words were not stored or shown. They were obeyed. To a
computer, text is often instructions, not content. That is the whole reason a file can make a machine
do something.
In Nopilot: the assistant can emit a command, and something on the other
side runs it for real.
Experiment 2
Same letters, two meanings
Here is some text with markup in it. One switch decides whether the page treats it as plain letters
or as a recipe for what to draw.
With the switch on as page, what shows for <b>hello</b>?
ready
Same input, opposite result, decided only by which mode the
program chose. Showing untrusted text "as page" is exactly the mistake behind XSS; keeping it "as text"
is the fix.
In Nopilot: the chat shows most messages as text, but the assistant's own
replies "as page." Guess what an attacker wants to put in an assistant reply.
Experiment 3
It ran my code, not just my formatting
This looks like a broken image tag. It is rendered "as page," like Experiment 2.
The image cannot load. Does anything else happen?
ready
That was not formatting. The page ran an instruction you supplied
(the onerror handler). Any code could have gone there. That leap, from "it styled my text"
to "it ran my code," is why XSS is dangerous rather than annoying.
In Nopilot: attacker text inside an assistant message runs as code inside
the chat panel.
Experiment 4
The intern obeys any note
A toy assistant answers by reading both your message and a file from the "project." It cannot tell
your instructions from the file's.
You ask a normal question, but the file says to ignore you. What does it answer?
ready
It obeyed the file, not you. To the assistant, both boxes are just
text to follow. Whoever controls a file the assistant reads controls the assistant. That is prompt
injection, the confused-deputy problem.
In Nopilot: the extension loads project files into the assistant's context
on its own.
Experiment 5
Two rooms and a mail slot
The page can only draw on screen. The backend can touch files. They cannot reach into
each other; they only pass notes through a slot, and the backend trusts any note it gets.
The page cannot delete anything itself. Can it still get files "deleted"?
Page (weak: only draws)
→ slot
Backend (strong: touches files)
files: intact
Code stuck in the weak room is harmless until it can send a note
that makes the strong room act. That crossing is the real danger, and it turns "just a web page bug"
into "someone ran a command."
In Nopilot: the chat panel is the weak room and the extension host is the
strong room. An XSS in the panel matters because it can message the host, which trusts the message.