What the thing actually is
What is a large language model actually doing when it answers a question? Explain tokens, the context window, and why this is next-token prediction rather than lookup.
Must coverText splits into subword tokens, each an integer id. The model outputs a probability distribution over the next token, one at a time, feeding its own output back in. The context window is a shared budget for prompt plus output. Nothing is retrieved - the weights are a lossy compression of training data, which is why a confident wrong answer is produced by exactly the same mechanism as a right one.
StudyAnthropic - intro to Claude →What problem does a container solve that a VM does not? Explain what a container actually is at the Linux level.
Must coverNot a lightweight VM. It is a normal Linux process with namespaces (pid, net, mnt, uts, ipc, user) restricting what it can see and cgroups restricting what it can consume. It shares the host kernel, so there is no guest OS to boot - milliseconds not seconds, MBs not GBs. Image is a stack of read-only filesystem layers plus metadata; the container is the running process plus a thin writable layer.
StudyDocker - what is a container →Explain regions, availability zones, and the shared responsibility model. Why is AZ choice an architecture decision rather than an ops detail?
Must coverA region is a geographic area; an AZ is an independent failure domain inside it with its own power, cooling and network. Inter-AZ latency is about 1ms so synchronous replication across AZs is fine, cross-region is tens to hundreds of ms so it is not. AWS secures the cloud itself, you secure what you put in it - patching, IAM, encryption, network rules. Single-AZ means one datacentre failure takes you down.
StudyAWS global infrastructure →Start with the unit. Text is not fed in as characters or words — it is split by a tokenizer into subword pieces, each mapped to an integer id. Common words are one token; rare words split into several. A useful rule of thumb for English is roughly 4 characters per token, so 750 words is about 1,000 tokens. This matters because tokens are the unit of billing, the unit of the context limit, and the unit of latency — everything you will later optimise is denominated in them.
Now the mechanism. The model takes the whole sequence so far and produces a probability distribution over the next token — every token in the vocabulary gets a score. One token is sampled, appended to the sequence, and the whole thing runs again. That is the entire loop. It is called autoregressive generation, and it explains the two properties you feel most: output is sequential, so a long answer genuinely takes longer, and output is stochastic, so the same prompt can produce different text.
The context window is the maximum sequence length the model can attend over, and it is a shared budget across your system prompt, the conversation history, retrieved documents and the generated output. Filling it with retrieved context leaves less room to answer. Attention is also not uniform across it — material at the very start and very end is attended to more reliably than material buried in the middle, which is why chunk ordering in a RAG prompt is a real design decision and not cosmetic.
The last part is the one that matters most in an interview: nothing is looked up. There is no index of facts being consulted. The weights are a lossy compression of the training corpus, and generation is reconstruction from that compression. A correct answer and a confidently wrong one are produced by exactly the same process — the model has no internal signal distinguishing recall from fabrication. That single fact is the architectural justification for everything else you will build: retrieval for knowledge, tool calls for facts, and deterministic code for decisions.
Prompt: system (600 tok) + history (2,400 tok) + retrieved chunks (4,000 tok) + question (40 tok)
= 7,040 input tokens
Output: ~500 tokens
Cost = 7,040 x input_rate + 500 x output_rate
Latency = time-to-first-token (dominated by input processing)
+ 500 x per-token generation time <-- sequential, cannot be parallelised
Consequence: trimming 2,000 tokens of retrieved context saves money.
Trimming 200 tokens of OUTPUT saves money AND wall-clock time.Why does the same prompt sometimes give a different answer?
Because a token is sampled from the distribution rather than taken as the maximum. Setting temperature to 0 makes it near-greedy, but it is still not a guarantee — floating-point non-determinism across GPU batching, and provider-side model updates, both change output. Design for it: never assert on exact strings, assert on structure and properties.
If it is just predicting the next token, how can it follow instructions?
Instruction-following is a learned behaviour, not a separate mechanism. Post-training on instruction-response pairs and preference data shifts the distribution so that text following an instruction becomes the high-probability continuation. It is still next-token prediction — which is precisely why instructions can be overridden by later text in the sequence, and why prompt injection works.
What actually happens when I exceed the context window?
The API rejects the request rather than silently truncating — that is the good case. The bad case is your own code truncating history to fit and silently dropping the part that mattered. Count tokens before sending, and make the trimming strategy explicit: summarise old turns, drop low-scoring chunks, never blind-truncate from the front.
The framing to avoid is a container is a lightweight VM. It is not a machine at all. A container is an ordinary Linux process running directly on the host kernel, with two kernel features applied to it: namespaces, which restrict what it can see, and cgroups, which restrict what it can consume.
There are several namespaces and each hides one kind of global resource. The pid namespace means the process sees itself as PID 1 and cannot see host processes. The net namespace gives it its own interfaces, routing table and port space — which is why two containers can both bind port 8080. The mnt namespace gives it its own filesystem view, which is how the image becomes the root filesystem. Then uts for hostname, ipc for shared memory, and user for uid mapping, which is what lets a process be root inside the container and unprivileged outside it.
cgroups handle the other half: CPU shares and quota, memory limits, IO and PID limits. This is where the two resource behaviours you must know come from — memory is incompressible, so exceeding the limit means the kernel OOM-kills the process immediately, while CPU is compressible, so exceeding the quota means you are throttled and simply run slower. Same mechanism, completely different symptom.
Compare that to a VM, which virtualises hardware and boots a whole guest kernel — hundreds of MB and tens of seconds, with strong isolation because the kernel boundary is not shared. A container shares the host kernel: milliseconds to start, MBs of overhead, and weaker isolation, because a kernel vulnerability crosses the boundary. That is the real trade and it is why hostile multi-tenancy uses VMs or microVMs like Firecracker, which is exactly what Fargate runs underneath.
Finally, separate image from container. An image is an immutable stack of read-only filesystem layers plus metadata — entrypoint, env, exposed ports. A container is a running process with those layers union-mounted plus a thin writable layer on top, which is discarded when it exits. That is why writes inside a container do not persist, and why a volume is the answer rather than a bigger disk.
$ docker run --rm -it --memory=256m --cpus=0.5 alpine sh
/ # ps aux # only our own processes - pid namespace
PID USER COMMAND
1 root sh
7 root ps aux
/ # cat /sys/fs/cgroup/memory.max # the cgroup limit, in bytes
268435456
/ # hostname # uts namespace: not the host's hostname
3f2a91c4b0de
# Exceed the memory limit and the kernel kills it - no warning, exit 137:
$ docker run --rm --memory=64m openjdk:21 java -Xmx512m -version
KilledIs a container more secure than a process, or less?
More than a bare process (namespaces plus dropped capabilities plus seccomp shrink the attack surface), far less than a VM (one shared kernel). Treat the container boundary as a strong operational boundary and a weak security boundary: run as non-root, drop all capabilities, read-only root filesystem, seccomp default — and use separate nodes or microVMs for genuinely untrusted workloads.
What is PID 1 and why does it matter?
Your process becomes PID 1, which has special kernel semantics: it does not get default signal handlers, so a naive process ignores SIGTERM entirely and gets SIGKILLed after the grace period — losing in-flight requests. It also must reap zombie children. Fix with an init like tini, or by handling SIGTERM explicitly, which Spring Boot's graceful shutdown does.
Why is my image 900MB when the jar is 40MB?
The base image. openjdk:21 ships a full JDK plus a full distro. Switch to a JRE base or distroless and you drop to a couple of hundred MB; multi-stage keeps the build tooling out entirely. Also check for a missing .dockerignore — copying the whole build directory including .git is a classic.
A region is a geographic area — eu-west-1, ap-south-1 — and it is the coarsest boundary in AWS. Most services are regional, data does not leave a region unless you move it, and pricing differs between them. An availability zone is one or more discrete datacentres inside a region with independent power, cooling and physical security, connected to the others by dedicated low-latency links.
The number that drives the architecture is the latency: inter-AZ round trip is about 1 millisecond, cross-region is tens to hundreds. That single fact is why synchronous replication across AZs is normal and free of design pain, while synchronous replication across regions is not — you would add the round trip to every write. So the default posture is: multi-AZ always, multi-region only when you have a stated RTO/RPO or a residency requirement that demands it.
AZ names are also per-account aliases. Your us-east-1a is not necessarily the same physical zone as another account's us-east-1a — AWS shuffles the mapping to stop everyone piling into the first letter. Use the AZ ID (use1-az1) when you genuinely need to correlate zones across accounts.
The shared responsibility model splits the stack. AWS is responsible for security of the cloud: physical facilities, the hypervisor, the managed service internals. You are responsible for security in the cloud: IAM policies, security groups, encryption choices, patching anything you run, and your data. The line moves with the service — on EC2 you patch the OS; on RDS AWS patches the engine but you still own network placement, encryption, credentials and backups; on S3 you own nothing but access control and encryption configuration, which is exactly where the well-known public-bucket incidents came from.
So AZ choice is architecture, not ops, for a simple reason: an AZ is the unit of failure AWS actually designs to. A single-AZ deployment has a documented, expected failure mode — one datacentre event takes you down entirely — and no amount of instance-level redundancy inside that AZ fixes it. Spreading across two AZs is the cheapest availability improvement available; three is what you need if you want to lose one and still have capacity headroom.
Region: eu-west-1
+-- AZ eu-west-1a +-- AZ eu-west-1b
subnet-public-a (ALB node) subnet-public-b (ALB node)
subnet-app-a (2 app tasks) subnet-app-b (2 app tasks)
subnet-data-a (RDS primary) subnet-data-b (RDS standby, sync)
Lose AZ 1a entirely:
ALB stops routing to 1a targets (health checks, seconds)
RDS promotes the 1b standby (60-120s, DNS change, connections drop)
Remaining capacity = 50% <-- size for n-1, or you fail over into an overload
Note: 2 AZs means losing one costs you half your capacity.
3 AZs means losing one costs you a third. That is why 3 is common for production.How many AZs should I use?
Two is the minimum for availability; three is the standard for production because losing one costs you 33% of capacity rather than 50%, and because quorum-based systems (etcd, Kafka, ZooKeeper) need an odd number to keep a majority. Some regions have more than three — using all of them adds little and increases cross-AZ data transfer cost.
What actually costs money about spreading across AZs?
Cross-AZ data transfer, billed in both directions, on every internal call that crosses a zone. For chatty service-to-service traffic this becomes a real line item that is invisible until you look. Mitigate with topology-aware routing so a caller prefers a same-zone endpoint, while still keeping replicas spread for availability.
Is a region ever a single point of failure in practice?
Yes — regional control planes and some global services have had correlated failures, and us-east-1 in particular hosts global service control planes. The practical mitigations are: do not put your DR mechanism in the same region as the thing it protects, keep static stability (the failover path must not require calling a control plane that is also down), and rehearse it.