From a simple idea to help my mother with her meeting notes to a full-fledged mobile application - here's the story of how my transcription tool evolved through different technologies and iterations.

The Beginning: A Tool for My Mother (2023)

It all started when I noticed my mother struggling with digital note-taking as a meeting secretary. Traditional dictation tools weren't cutting it - they produced poorly structured text that was difficult to follow. This inspired me to build a solution using React, OpenAI's Whisper API, and GPT-3.5.

Initial Technical Stack

• Frontend: React for its component-based architecture

• AI Integration: OpenAI's Whisper API for accurate speech-to-text

• Enhancement: GPT-3.5 for improving text structure and speaker identification

The prototype was a success, significantly reducing her post-meeting workload and even proving valuable for user testing sessions. However, there was one major issue: the exposed API key in the front end made it vulnerable.

The Chrome Extension Phase (Autumn 2024)

To address the security concerns of the initial tool and add monetization capabilities, I developed a Chrome extension version with:
- Secure backend architecture in Node.js
- Protected API integrations
- In-app purchase capabilities

Notely (Chrome extension)

Architectural Improvements

• Backend: Node.js server for secure API key management

• Payments: Integration with Stripe for in-app purchases

The iOS App (December 2024)

The journey came full circle when I decided to rebuild the transcription tool as an iOS application using Flutter. This decision was natural given my earlier exploration with the framework. The existing backend architecture from the Chrome extension provided a solid foundation.

Some key factors that influenced choosing Flutter:

• Fast development cycle

• Excellent UI customization

• Strong performance

• Prior experience with Dart

• Support for cross-platform development

Development Process

The development leveraged Claude 3.5 Sonnet for programming assistance and implemented essential features like:

• Audio file handling and WAV conversion

• Chunked file processing

• Secure API integration

• Local database storage

• Clean, intuitive UI

This journey showcases how a simple solution for one person's problem evolved into a fully-featured application, incorporating various technologies and learning experiences.

Prompt to Claude:

create a flutter application that you can run on ios and upload voice files which are transcribed. It should chunk the audio files and convert to wav files. You can have a look at this chrome extension i built in react.js. Please do it step by step: 
I also want the text to be saved in a DB. 

Answer from Claude:

I'll help you create a Flutter application for audio transcription, incorporating the key functionality from your Chrome extension. Here's a step-by-step approach:

Let's go through the implementation steps 🚀:

1. Project Setup:

• Create Flutter project:

  flutter create audio_transcriber

• Add dependencies in pubspec.yaml:

flutter_ffmpeg: ^0.4.2  
path_provider: ^2.0.15  
sqflite: ^2.3.0  
http: ^1.1.0

// lib/models/audio_chunk.dart
class AudioChunk {
  final int offset;
  final int size;
  final List<int> data;

AudioChunk({
    required this.offset,
    required this.size,
    required this.data,
  });
}

// lib/services/audio_processor.dart
import 'dart:typed_data';
import 'package:path_provider/path_provider.dart';
import 'package:flutter_ffmpeg/flutter_ffmpeg.dart';

class AudioProcessor {
  static const int CHUNK_SIZE = 5 * 1024 * 1024; // 5MB chunks
  final FlutterFFmpeg _flutterFFmpeg = FlutterFFmpeg();

Future<List<AudioChunk>> processAudioFile(String filePath) async {
    final file = File(filePath);
    final bytes = await file.readAsBytes();
    final chunks = <AudioChunk>[];
    
    // Extract header based on file extension
    final header = await _extractAudioHeader(bytes, filePath);
    
    // Process chunks
    int offset = header.offset;
    while (offset < bytes.length) {
      final end = min(offset + CHUNK_SIZE, bytes.length);
      final chunkData = bytes.sublist(offset, end);
      
      // Create chunk with header
      final fullChunk = Uint8List(header.size + chunkData.length);
      fullChunk.setAll(0, header.data);
      fullChunk.setAll(header.size, chunkData);
      
      chunks.add(AudioChunk(
        offset: offset,
        size: fullChunk.length,
        data: fullChunk,
      ));
      
      offset = end;
    }
    
    return chunks;
  }

Future<String> convertToWav(String inputPath) async {
    final dir = await getTemporaryDirectory();
    final outputPath = '${dir.path}/converted.wav';
    
    await _flutterFFmpeg.execute(
      '-i $inputPath -acodec pcm_s16le -ar 44100 -ac 2 $outputPath'
    );
    
    return outputPath;
  }
}

// lib/services/transcription_service.dart
class TranscriptionService {
  static const String API_ENDPOINT = 'https://your-api-endpoint/transcribe';
  
  Future<String> transcribeChunk(AudioChunk chunk, int index, int total) async {
    final uri = Uri.parse(API_ENDPOINT);
    final request = http.MultipartRequest('POST', uri)
      ..fields['chunkIndex'] = index.toString()
      ..fields['totalChunks'] = total.toString()
      ..files.add(
        await http.MultipartFile.fromBytes(
          'audioChunk',
          chunk.data,
          filename: 'chunk_$index.wav',
        ),
      );
final response = await request.send();
    if (response.statusCode != 200) {
      throw Exception('Failed to transcribe chunk');
    }
    
    final responseData = await response.stream.bytesToString();
    return json.decode(responseData)['transcription'];
  }
}

// lib/db/database.dart
class TranscriptionDatabase {
  final Database _db;
  
  Future<void> saveTranscription(String audioPath, String transcription) async {
    await _db.insert('transcriptions', {
      'audio_path': audioPath,
      'transcription': transcription,
      'created_at': DateTime.now().toIso8601String(),
    });
  }
}

Key Features Implemented 🎯:

• Audio file selection and recording

• Chunking large audio files

• WAV conversion using FFmpeg

• API integration for transcription

• Local database storage

Then I continued with each step until it worked on my phone and from there it started.

Audio Processing Challenges 🎤

The foundation of the app rested on robust audio handling. Initially simple in concept, this became one of our most complex technical challenges:

• Recording Conflicts: Separating recording and playback systems to prevent resource conflicts

• Zero-Length Recordings: Debugging timing issues causing empty audio files

• Session Management: Handling audio session transitions between recording and playback

• Permission Handling: Implementing robust microphone permission flows

Cloud Architecture Evolution ☁️

The backend infrastructure evolved significantly during development:

• Authentication: Implementing both Google and Apple Sign-In

• Data Sync: Building a reliable cross-device synchronization system

• Error Handling: Developing robust error recovery for offline/online transitions

• Firebase Integration: Resolving Firestore permission issues and deployment conflicts

iOS-Specific Hurdles 📱

The iOS ecosystem presented its own set of challenges:

// Key areas requiring attention:
- CocoaPods configuration
- Deployment target conflicts
- Background audio mode setup
- In-app purchase implementation

App Store Preparation 🎯

Getting ready for the App Store involved multiple steps:

Documentation

• Privacy policy creation

• Terms of service documentation

• Subscription details and pricing strategy

Technical Requirements

• TestFlight distribution setup

• App Store Connect configuration

• Build number management

• Screenshot generation

Testing Strategy

1. User acceptance testing

2. Performance optimization

3. Subscription flow verification

4. Error scenario testing

Key Learnings 🎓

Several critical insights emerged during development:

1. Audio Architecture
   - Separate recording and playback systems
   - Robust error handling for audio sessions
   - Careful state management for transcription status

2. Cloud Sync
   - Implement offline-first architecture
   - Handle cross-device deletion conflicts
   - Maintain data integrity during sync

3. User Experience
   - Clear feedback during processing
   - Intuitive error messages
   - Smooth subscription flow

Looking Forward 🚀

Future development plans include:
- Multi-language support
- Advanced text analysis
- Real-time collaboration features
- Enhanced accessibility features

This journey from concept to App Store submission highlighted the importance of thorough planning, robust architecture, and careful attention to platform-specific requirements. Each challenge solved brought valuable insights for future development.

Launch on App Store

Technical Learnings and Insights

Flutter vs React Native

After working with both frameworks, Flutter stood out for:

• More predictable performance

• Better developer tooling

• Simpler state management

• Extensive widget library

Architecture Evolution

The project evolved from a simple client-side application to a full-stack solution with:

• Microservices architecture

• Event-driven design

• Caching strategies

• Error handling and recovery