Data quality determines model quality. Collecting data Identify sources, check permissions and licensing, and ensure you can legally use the data. Cleaning Handle missing values, duplicates, inconsistent formats and outliers. Labelling For supervised learning, labels must be accurate and consistent. Write labelling guidelines and check agreement between labellers. Splitting Split data before any analysis that […]

Large language models generate and transform text and can be accessed through APIs. Key concepts Tokens: units of text that determine cost and context limits. Context window: the maximum input and output size. Temperature: controls randomness. System and user messages: set behaviour and provide requests. Integration basics Send requests from server-side code, keep API keys […]

Prompt design strongly affects output quality. Principles Be specific about the task, audience and format. Provide context and relevant data. Give examples of desired output. Break complex tasks into steps. State constraints, such as length and tone. Structured output Ask for JSON matching a schema, validate it in code and handle invalid responses. Many APIs […]

Retrieval-augmented generation (RAG) grounds model answers in your own documents. The pipeline Split documents into chunks. Create embeddings for each chunk. Store them in a vector index. Embed the user’s question and retrieve relevant chunks. Provide chunks to the model with instructions to answer from them. Chunking Chunk size affects retrieval. Keep sections meaningful and […]

Models can call tools, such as search, calculators or APIs, to complete tasks. Tool calling Define tools with clear names, descriptions and parameter schemas. The model chooses a tool and your code executes it. Agents Agents loop: plan, call tools, observe results and continue until done. They can handle multi-step tasks but are harder to […]

AI systems need systematic evaluation because outputs vary. Evaluation sets Create a representative set of inputs with expected outputs or criteria. Include edge cases and adversarial inputs. Metrics Exact match, schema validity, retrieval accuracy, factual consistency with sources, and human ratings for quality. Automated grading Code-based checks or model-based graders can scale evaluation; validate graders […]

Deploying AI features requires attention to reliability, cost, safety and ethics. Deployment Put AI calls behind service interfaces, add timeouts and fallbacks, cache results and monitor latency and errors. Monitoring Track usage, cost, quality signals, user feedback and drift in input data. Responsible AI Be transparent with users that AI is used. Protect privacy and […]