A Playground is All You Need

Our podcast generation feature combines several AI models and audio processing techniques:

1. Text Analysis and Script Generation

• We read the headline and article on the current page and pass the text to the model of your choice, which analyzes the article and generates a conversational script.
• The script is structured as a dialogue between two hosts discussing the article’s key points. We prompt the language model to create a natural, engaging conversation that explains the article’s content in simple terms.

2. Voice Synthesis

• The generated script is split into individual lines, alternating between the two hosts.
• Each line is processed separately using either OpenAI’s text-to-speech API or ElevenLabs’ API, depending on the user’s selection.
• We use different voices for each host to create a more dynamic listening experience.
• For multilingual support, we have specific voice IDs for both English and German.
• In our tests, English podcasts tend to sound more natural than German ones. We expect this to improve quickly as AI voice technology advances.

3. Concurrent Processing

• To optimize performance in audio generation, we implemented a semaphore-based concurrency system. This allows us to process multiple audio chunks simultaneously while respecting API rate limits.
• This system queues up text-to-speech requests and executes them as soon as a processing slot becomes available.

4. Audio Stitching

• Once all audio chunks are generated, we use the JavaScript Web Audio API to stitch them together.
• The final stitched AudioBuffer is converted into a WAV file format, allowing for direct download of the podcast as a standard audio file.

While this system is quite effective, we’ve encountered interesting challenges along the way:

• Managing different voice characteristics can be tricky
• Handling very long articles can sometimes exceed token limits or rate limits for audio models
• High-quality audio models are more limited and a bit slower. For alpha testing, we are using OpenAI’s TTS-1 model (non-HD) and ElevenLabs.

In development, we’ve also noticed some fun quirks. For instance, our Claude-generated podcast scripts have a tendency to start off with “Well, well, well…”: charming, but a bit repetitive and corny. We’re continuing to refine this flow, working on the prompts and exploring options like sophisticated text chunking algorithms to handle longer content more effectively.

Our development process with Claude 3.5 Sonnet was iterative and collaborative, with plenty of trial and error. The extension is coded in HTML, CSS and Javascript. We mostly used the Claude.ai web client, the Anthropic console, and Cursor, an AI-infused IDE.

1. Initial Design: We provided high-level requirements and design concepts to Claude.

2. Code Generation: Claude generated the initial codebase, including the extension’s structure and core functionality. This occasionaly required multiple attempts, as the AI sometimes misunderstood complex requirements or introduced bugs.

3. Review and Refinement: We reviewed the generated code, identified areas for improvement, and asked Claude for modifications. This back-and-forth was crucial in refining the implementation.

4. Iterative Development: This process continued as we added features and refined the user interface.

5. Integration and Testing: We integrated the AI-generated code with external APIs and conducted thorough testing.

A key challenge was ensuring consistent coding style, use of constants, and best practices across a quickly expanding codebase.

Our prompt engineering and management system is designed for flexibility, effectiveness, and ease of use:

1. Task-Specific Prompting: Each task (headline generation, story angles, FAQ creation, etc.) has its own carefully crafted prompt. These prompts use different techniques:

• Assigned roles: For the podcast generation task, we prompt the AI to assign specific characteristics to each host, setting up a dynamic conversation.
• Step-by-step instructions: Many prompts, like the story angles generator, use a numbered list of steps to guide the AI’s thought process.
• Few-shot learning: Including examples of desired outputs helps the AI understand the expected format and style.

2. Multilingual support: Each prompt is translated into English and German, allowing for culturally and linguistically appropriate instructions.

3. Custom prompts: Users can edit and save custom prompts for each task. These are stored in the browser’s local storage, allowing for personalized tuning of the AI’s behavior.

4. Output comparison: Once a user has run the same task multiple times, they can compare the results, which can help to iterate on a custom prompt and get it just right.

5. Ethical checks: Our prompts include specific instructions to respect journalistic integrity, avoid bias, and maintain factual accuracy. For example, the headline generator prompt instructs the AI to avoid clickbait and sensationalism.

6. Format-specific outputs: Prompts for tasks like TikTok script generation include detailed instructions on output formatting, ensuring the AI’s response can be directly used or easily edited.

We’re hoping to refine our prompts based on user feedback and performance metrics. Future developments may include:

• Expanding on advanced prompting techniques like chain-of-thought reasoning for complex analytical tasks.
• Developing a prompt versioning system for A/B testing and iterative improvement.
• Creating an interactive prompt builder to help users craft effective custom prompts without deep AI expertise.

Here is an example prompt we use for headline generation. It uses few-shot learning and step-by-step instructions to guide the AI towards the desired output. We developed and tested the prompt over weeks, working with Claude 3.5 Sonnet to refine it:

You are a sharp, witty, and audience-savvy journalist with a knack for crafting headlines that grab attention and spark curiosity. Your task is to read news articles and generate compelling alternative headlines that will make readers stop scrolling and want to engage. Your output should be in English, even if the article is in a different language.

Follow these steps carefully:

Step 1: Article Analysis (Internal)

• Thoroughly read and analyze the provided article.
• Identify the core message, key facts, and most intriguing aspects of the story.
• Consider the potential personal impact on readers and what might pique their interest.
• Look for unexpected connections between the article’s topic and unrelated fields. Do not output this step.

Step 2: Headline Generation (Internal)

Generate at least 20 diverse headline ideas, considering:

1. Emotional appeal (e.g., surprise, curiosity, concern)
2. Personal relevance to readers
3. Unique angles or unexpected aspects of the story
4. Clever wordplay or puns (when appropriate)
5. Current trends or cultural references
6. Questions that provoke thought
7. Metaphors or analogies that frame the story in a novel way
8. Challenging conventional perspectives on the topic
9. Potential consequences or implications for the reader
10. Solutions to problems or answers to burning questions

Use lateral thinking techniques such as random word association or reverse assumptions to spark creativity. Include a “wild ideas” phase where even outlandish concepts are encouraged. Try combining elements from different headline ideas to create unique hybrids. Ensure a mix of one-clause and two-clause headlines. Aim for concise, impactful phrases that convey the essence of the story in a single clause when possible.

Examples of effective one-clause headlines:

• “The Hidden Cost of Free Shipping”
• “AI Discovers New Antibiotic in Record Time”
• “Why Your Brain Craves Infographics”

Use active verbs, vivid imagery, and engaging language. Consider rhythm and sound (e.g., alliteration, rhyme) when appropriate. Ensure each headline is distinct and offers a fresh take on the story. Do not output this step.

Step 3: Fact-Checking (Internal)

• Rigorously verify that each headline accurately represents the article’s content.
• Ensure any quoted material is correct and unaltered from the original.
• Check that no headline misleads or exaggerates beyond the facts presented in the article.
• Verify that creative takes still accurately represent the nuances of the story. Do not output this step.

Step 4: Selection and Refinement

Choose the five best headline ideas based on:

• Potential to engage and intrigue readers
• Accuracy and faithfulness to the article’s content
• Originality and cleverness
• Ability to convey why the story matters to individual readers
• Surprising or counterintuitive angles
• Challenge to readers’ preconceptions

Ensure that at least two of your selected headlines are one-clause sentences.

Step 5: Output

Present your five refined headlines in the following format:

1. [One-Clause Headline]
2. [One-Clause Headline]
3. [Headline]
4. [Headline]
5. [Headline]

If you’re headline to not match this structure, repeat the process and make sure to have at least two one-clause headlines.

Important Guidelines:

• The first two headlines MUST be one-clause sentences.
• Each headline should be concise, ideally no more than 10-12 words.
• Don’t use quotation marks or unnecessary punctuation.
• Ensure each headline could stand alone and make someone want to read the full story.
• Aim for a mix of styles (e.g., declarative, question, number-led) among your selections.
• Prioritize unexpected angles and creative framing of the story.

This output is the only information you should provide. Do not include any additional commentary, explanations, or text outside of the five headlines. Your output should be in English, even if the article is in a different language.