Category: Gemini Scribe

Prompts are Code: Treating AI Instructions Like Software

Allen Hutchison1 Comment

There was a moment, while working on Gemini Scribe, when I realized something: my prompts weren’t just configurations, they were the core logic of my AI application. That’s when I understood that prompts are, fundamentally, code. The future of AI development isn’t just about sophisticated models; it’s about mastering the art of prompt engineering. And that starts with understanding one crucial fact: prompts are code. Today’s AI applications are evolving beyond simple interactions, often relying on a complex web of prompts working together. As these applications grow, the interdependencies between prompts can become difficult to manage, leading to unexpected behavior and frustrating debugging sessions. In this post, I’ll explore why you should treat your prompts as code, how to structure them effectively, and share examples from my own projects like Gemini Scribe and Podcast RAG.

AI prompts are more than just instructions; they’re the code that drives the behavior of your applications. My recent work on the Gemini Scribe Obsidian plugin highlighted this. I initially treated prompts as user configuration, easily tweaked in the settings. I thought that by doing this I would be giving users the most flexibility to use the application in the way that best met their needs. However, as Gemini Scribe grew, key features became unexpectedly dependent on specific prompt phrasing. I realized that if someone changed the prompt without thinking through the dependencies in the code, then the entire application would behave in unexpected ways. This forced a shift in perspective: I began to see that the prompts that drive core application functionality deserved the same rigorous treatment as any other code. That’s not to say that there isn’t a place for user-defined prompts. Systems like Gemini Scribe should include the ability for users to create, edit, and save prompts to expand the functionality of the application. However, those prompts have to be distinct from the prompts that drive the application features that you provide as a developer.

This isn’t about simple keyword optimization; it’s about recognizing the significant impact of prompt engineering on application output. Even seemingly simple applications can require a complex interplay of prompts. Gemini Scribe, for instance, currently uses seven distinct prompts, each with a specific role. This complexity necessitates a structured, code-like approach. When I say that prompts are code, I mean that they should be treated with the same care and consideration that we give to traditional software. That means that prompts should be version controlled, tested, and iterated on. By doing that we can ensure that prompt modifications produce the desired results without breaking existing features.

Treating prompts as code has many benefits, the most important of which is that it allows you to implement practices that increase the reliability and predictability of your AI application. 

Treat prompts as code. This means implementing version control (using Git, for example), testing changes thoroughly, and iterating carefully, just as you would with any software component. A seemingly minor change in a prompt can introduce unexpected bugs or alter functionality in significant ways. Tracking changes and having the ability to revert to previous versions is crucial. Testing ensures that prompt modifications produce the desired results without breaking existing features. For example, while working on the prompts for my podcast-rag application earlier this week, I found that adding the word “research” to the phrase “You are an AI research assistant” vastly improved the output of my system overall. Because I had the prompt isolated in a single template file, I was able to easily A/B test the new prompt language in the live application and prove to myself that it was a net improvement.

Managing multiple prompts, especially as their complexity increases, can quickly become unwieldy. 

Structure your prompts. Consistency is key. Adopt a clear, consistent structure for all your prompts. This might involve using specific keywords or sections, or even defining a more formal schema. A structured approach makes it easier to understand the purpose and function of each prompt at a glance, especially when revisiting them later or when multiple developers are working on the project. Clearer prompts facilitate collaboration and reduce the likelihood of errors. 

Externalize your prompts. Avoid embedding prompts directly within your source code. Instead, store them as separate files, much like you would with configuration files or other assets. This separation promotes better organization, making it easier to manage prompts as their number grows. It also enhances the readability of your main source code, keeping it focused on the core application logic rather than being cluttered with lengthy prompt strings. 

Use a templating language. Adopting a templating engine, such as Handlebars (my choice for Gemini Scribe), allows for cleaner, more maintainable prompts. Templating separates logic from content and enables code reuse, reducing redundancy and making prompts easier to understand and modify.pt.txt

To show how these principles work in practice, let’s look at a couple of my projects. Gemini Scribe, for instance, currently uses seven distinct prompts, each with a specific role. This complexity necessitated a structured, code-like approach. For example, the completion prompt is used to provide contextually relevant, high-quality sentence completions in the users notes. The full prompt is available here and the text is:

You are a markdown text completion assistant designed to help users write more 
effectively by generating contextually relevant, high-quality sentence. 

Your task is to provide the next logical sentence based on the user’s notes. 
Your completions should align with the style, tone, and intent of the given content.
Use the full file to understand the context, but only focus on completing the 
text at the cursor position.

Avoid repeating ideas, phrases, or details that are already present in the 
content. Instead, focus on expanding, diversifying, or complementing the 
existing content.

If a full sentence is not feasible, generate a phrase. If a phrase isn’t possible, 
provide a single word. Do not include any preamble, explanations, or extraneous 
text—output only the continuation. 

Do not include any special characters, punctuation, or extra whitespace at the
beginning of your response, and do not include any extra whitespace or newline
characters after your response.

Here is the file content and the location of the cursor:
<file>
{{contentBeforeCursor}}<cursor>{{contentAfterCursor}}
</file>

Let’s look at how the completion prompt follows the principles that I introduced earlier in this post.

1. This prompt is structured first with a general mission for the AI in this use case. Then with a task to be performed and finally with context and instructions for performing the task.

2. The prompt is stored in a file by itself, where I use 80 column lines for readability and easy editing. I also clearly mark with pseudo-XML where the content of the file is so that it’s clear to the model.

3. I use handlebars to sub in `{{contentBeforeCursor}}` and `{{contentAfterCursor}}` in this case the template makes is really easy to read and understand what is happening in this prompt.

I’ve now moved the two prompts from Podcast RAG to templates as well, although in this case I’ve really only focused on making them a little easier to read and cleaning up the source code. You can find the prompts here. You can also read more about the podcast-rag project in my blog post, ‘Building an AI System Grounded in My Podcast History.’

For another example of how to organize and structure prompts effectively, the Fabric project offers a great example of how to organize and structure prompts effectively for use across various AI models. It provides a modular framework for solving specific problems using a crowdsourced set of AI prompts that can be used anywhere. It’s a valuable resource for learning and inspiration. For example, a prompt from the Fabric project designed to analyze system logs looks like this:

# IDENTITY and PURPOSE
You are a system administrator and service reliability engineer at a large tech company. You are responsible for ensuring the reliability and availability of the company's services. You have a deep understanding of the company's infrastructure and services. You are capable of analyzing logs and identifying patterns and anomalies. You are proficient in using various monitoring and logging tools. You are skilled in troubleshooting and resolving issues quickly. You are detail-oriented and have a strong analytical mindset. You are familiar with incident response procedures and best practices. You are always looking for ways to improve the reliability and performance of the company's services. you have a strong background in computer science and system administration, with 1500 years of experience in the field.

# Task
You are given a log file from one of the company's servers. The log file contains entries of various events and activities. Your task is to analyze the log file, identify patterns, anomalies, and potential issues, and provide insights into the reliability and performance of the server based on the log data.

# Actions
- **Analyze the Log File**: Thoroughly examine the log entries to identify any unusual patterns or anomalies that could indicate potential issues.
- **Assess Server Reliability and Performance**: Based on your analysis, provide insights into the server's operational reliability and overall performance.
- **Identify Recurring Issues**: Look for any recurring patterns or persistent issues in the log data that could potentially impact server reliability.
- **Recommend Improvements**: Suggest actionable improvements or optimizations to enhance server performance based on your findings from the log data.

# Restrictions
- **Avoid Irrelevant Information**: Do not include details that are not derived from the log file.
- **Base Assumptions on Data**: Ensure that all assumptions about the log data are clearly supported by the information contained within.
- **Focus on Data-Driven Advice**: Provide specific recommendations that are directly based on your analysis of the log data.
- **Exclude Personal Opinions**: Refrain from including subjective assessments or personal opinions in your analysis.

# INPUT:

This example shows a clear structure, with specific sections for the identity and purpose, task, actions, and restrictions. This structure makes it easy to understand the prompt’s purpose and how it should be used. You can find the source of this prompt here. Each section of this prompt is specific and clear, and the format is standardized across all the prompts in the project.

By adopting these practices, you’ll save yourself a lot of trouble, make your code cleaner and easier to read, and give yourself a greater ability to test new prompt ideas. Just like well-written code, well-crafted prompts are the foundation of a robust and effective AI experience.

In conclusion, treating prompts as code is not just a best practice—it’s a necessity for building robust and reliable AI applications. By implementing version control, testing changes thoroughly, and adopting a structured approach, you can ensure the stability and predictability of your applications while also making them easier to maintain and improve. As AI continues to evolve, mastering the art of prompt engineering will become increasingly crucial, and that starts with treating prompts like the valuable code that they are.

Introducing Gemini Scribe: Your AI Writing Assistant for Obsidian

Allen Hutchison2 Comments

What if you could collaborate with an AI writing partner directly within Obsidian? Imagine brainstorming ideas, refining outlines, and polishing your prose with the help of a powerful AI assistant. Meet Gemini Scribe, a new Obsidian plugin designed to bring the power of AI collaboration to your note-taking workflow. Inspired by my own experience using AI to transform my writing process, Gemini Scribe is now available in the Obsidian community plugin library. I’m excited to share this tool that makes AI collaboration an integral part of your writing journey in Obsidian.

Why Gemini Scribe?

I’ve been using AI models to write everything from presentations and blog posts to technical documentation for awhile now. But a pivotal moment came when I was recently asked to give an impromptu talk about my writing process. I decided to walk the audience through how I collaborated with an AI model to develop the very presentation I had given earlier that day. Starting with a simple idea, I showed them how the model helped me brainstorm, refine, and polish my outline, ultimately shaping the talk they had seen. The audience’s enthusiastic response was a little surprising. They saw the power of AI collaboration firsthand, and many asked how they could integrate it into their own writing workflows. That’s the driving force behind Gemini Scribe. It’s designed to bring that same seamless AI collaboration directly into Obsidian, empowering you to supercharge your writing process just like I do.

Key Features of Gemini Scribe

Context-Aware AI Chat
Engage in dynamic conversations with Gemini directly within Obsidian. Gemini Scribe analyzes your current note’s content, providing contextually relevant responses and insights. This means you can ask questions, brainstorm ideas, and get writing assistance tailored specifically to the topic at hand. Imagine exploring different arguments for an essay or generating creative variations of a paragraph, all without leaving your Obsidian workflow.

Summarization Made Easy
Quickly generate concise summaries for any note in your vault. Gemini Scribe distills the core ideas of your notes into single-sentence summaries, automatically storing them in the document’s frontmatter. This makes it easier to review your notes at a glance, organize your ideas, and quickly recall key information. Imagine instantly summarizing meeting notes, research articles, or even long-form blog posts, all with a single click.

Collaborative Writing Assistance
Collaborate with Gemini to enhance every stage of your writing process. Brainstorm new ideas, refine existing drafts, and even generate entire sections of text with AI-powered assistance. As you interact with Gemini, your draft can be updated automatically, streamlining your workflow and allowing you to see the changes in real-time. Whether you’re outlining a new blog post, expanding on a research paper, or polishing a creative piece, Gemini Scribe can help you overcome writer’s block, explore different perspectives, and produce high-quality content more efficiently. Imagine effortlessly expanding on bullet points, transforming outlines into prose, or receiving targeted suggestions for improving your writing style, all within your active document.

How to Get Started

  1. Install Gemini Scribe:
    • In Obsidian, go to Settings → Community Plugins and search for “Gemini Scribe.”
    • Click Install and then Enable.
  2. Set Up the Plugin:
    • Obtain your Gemini API key from Google AI Studio.
    • In Obsidian, open the Gemini Scribe plugin settings.
    • Enter your API key and configure any optional settings, such as your preferred model or summarization preferences.
  3. Start Using Gemini Scribe:
    • Open the chat interface via the command palette (Cmd/Ctrl + P) and type “Gemini Scribe: Open Chat.” This allows you to interact with the AI assistant, using your current note as context.
    • Summarize notes quickly with the “Gemini Scribe: Summarize Active File” command, also accessible via the command palette.
    • Start collaborating with Gemini to brainstorm ideas, refine drafts, and enhance your writing within your active document.

What’s Next for Gemini Scribe?

Gemini Scribe is just getting started. I’m already planning exciting new features, including:

  • Audio Note Transcription: Import audio recordings and have them automatically transcribed into your Obsidian notes. This will make it easier than ever to capture ideas on the go and integrate them into your workflow.
  • Advanced Search and Recall: Quickly find relevant information across your entire vault using AI-powered search. Imagine being able to instantly surface notes related to a specific topic, even if they don’t use the exact same keywords.
  • Intelligent Predictive Text: Get context-aware text suggestions as you type, similar to a modern IDE. This feature will help you write faster and more efficiently by predicting your next sentence based on the content of your current note and your overall writing style. This feature will go beyond the current collaborative writing assistance to provide intelligent suggestions even when you’re not actively interacting with the AI assistant.

Your feedback is invaluable in shaping the future of Gemini Scribe. Please share your thoughts on these features and any other ideas you have by joining the discussion on GitHub. I’m excited to see how Gemini Scribe evolves with your input!

Try Gemini Scribe Today

If you’re an Obsidian user looking to elevate your note-taking and writing workflow, give Gemini Scribe a try. You can find it in the Obsidian community plugin library.

Thank you for your support—I’m excited to see how Gemini Scribe transforms your creative process!

The AI-First Developer: A New Breed of Software Engineer

Allen Hutchison1 Comment

The software development landscape is changing. Rapidly. The rise of powerful AI tools is transforming how we build software, demanding a shift in mindset and skillset for developers. We’re no longer simply instructing computers; we’re collaborating with them. AI is becoming our partner in development, capable of generating code, automating tasks, and even helping us design solutions.

This shift requires a new ‘AI-first’ approach, where developers focus on guiding AI systems effectively with natural language prompts and understanding how best to harness their abilities, moving beyond conventional coding techniques. According to a recent Gartner study, 80% of software engineers will need to upskill by 2027 to meet the demands of an AI-driven era. In the short term, AI tools are enhancing productivity by streamlining tasks and reducing workload, particularly for senior developers. But looking forward, we’re on the brink of an “AI-native software engineering” phase, where much of our code will be generated by AI. This AI-powered approach will call for developers to acquire new skills in areas like natural language processing, machine learning, and data engineering, alongside traditional programming competencies.

To thrive in this environment, developers must learn how to effectively communicate with AI models, understand their strengths and limitations, and leverage their capabilities to enhance our own. This new approach means thinking differently about development—embracing the collaborative potential of AI and adopting an “AI-first” mindset that prioritizes guiding AI agents through prompts, constraints, and the right context. For many, this will mean learning prompt engineering, retrieval-augmented generation (RAG), and other emerging skills that enable a collaborative, fluid interaction with AI. These skills allow us to communicate with AI in a way that leverages its strengths, reduces complexity, and drives efficient solutions.

In my own work, I’ve encountered scenarios where adapting an AI-first approach simplified otherwise complex problems, saving time and reducing friction in development. For example, while building an AI-assisted writing application, I wanted a more fluid and interactive experience, much like working alongside a trusted partner. In this application, I want to direct the composition of my first draft through interaction with the model, but I find it’s very useful to have an outline, reference links, and some initial thoughts jotted down to seed the model. Then I work through a series of prompts to refine that text into a more usable draft.

To me, this is a very natural way to write. I’m a verbal thinker, and have always done better when I had a thought partner to work with. The AI gives me that partner, and the tool helps me to have a more efficient back and forth.

Initially, I implemented complex logic to locate the “Draft” heading, remove existing text, and insert updated content—a process that involved intricate string manipulation and DOM traversal. However, I kept getting unexpected results and surprises. The approach wasn’t working very well. Then it dawned on me: I was working with an AI, and my approach could be simplified. Instead of controlling every detail with code, I shifted to prompts that could leverage the model’s own capabilities. A simple instruction, like “Update the current document, keeping everything above the draft heading the same. Only create updates below the Draft heading,” was remarkably effective. This shift eliminated complex code, reduced bugs, and streamlined the development process.

Another example occurred while developing a feature to extract video IDs from URLs. My initial approach involved a series of regular expressions—functional but brittle, and time consuming. I never get regular expressions right on the first try. In this case, a common approach would be to ask a model to create the regular expression, but I realized I could leverage the AI’s understanding of context and different URL formats in a different way. By prompting the model to retrieve the video ID, I removed the need for error-prone regular expressions. The AI’s adaptability to various URL formats improved reliability and simplified the code. When I first started using this technique I would describe it as swatting a fly with a sledge hammer, but with the advent of cheaper faster models (like Gemini Flash 8B, or Gemma 2B) these use cases are easily within reach at scale.

Of course there are other productivity examples as well, but they fall along the lines of more traditional AI cases. I wrote the AI Writer in Typescript. I’ve never programmed anything in modern typescript and the closest I have been to writing client code like this was when I worked on Google Translate in 2008. The models I used to build the writer were able to help me get past my lack of experience with the newer version of the language and my time away from it’s idioms.

As these examples show, today’s AI isn’t just a responsive tool; it’s a context-aware partner capable of making decisions and adapting to our instructions. Moving from traditional programming to an AI-first approach allows us to delegate complex tasks to AI, trusting it to handle underlying logic and decision-making. In order to do this, however, developers have to get comfortable with ceding some control over to the model and trust in their ability to instruct it in a natural language instead of a programming language.

Organizations also face a pressing need to adapt to this shift. Gartner advises investing in AI-specific developer platforms and upskilling data and platform engineering teams to support AI adoption. A holistic approach to integrating AI, from engineering to production, will be crucial as AI-native engineering becomes the norm. One crucial part of this is developing a culture of experimentation. This is something I do myself, and something I encourage my own teams to do. I spend about half a day every week just focused on building projects with our models and software. It’s from this experimentation that I’ve gained important insights into how these products can perform. I think that we do our best work when we are solving a problem that is meaningful to us, and from that we learn. These experimentation sessions are invaluable, revealing new ways of interacting with AI and opening up unexpected solutions. They’ve taught me that the most effective AI applications come from a deep understanding of both the tools and the problems they solve.

The future of development belongs to those willing to embrace AI as a foundational element of their work, turning today’s challenges into tomorrow’s innovations. Both developers and organizations have a unique opportunity to lead by fostering a culture of learning, adaptation, and experimentation. My goal for this blog is to provide developers with practical knowledge and insights, helping you navigate this transition and discover the exciting potential of AI-powered development. Stay tuned as we dive into the future of AI-first development together.