Skip to content

Leveraging Flutter Gemma: Bringing Advanced AI to Your Flutter Application

  • by

Introduction

Flutter Gemma enables developers to incorporate Google’s advanced Gemma language models directly into their iOS and Android applications. This guide will walk you through the installation, setup, and usage of the Flutter Gemma package, providing unique code examples and culminating in a mini-project to help solidify your understanding.

Features

  • Local Execution: Run Gemma models directly on user devices, ensuring enhanced privacy and offline functionality.
  • Platform Support: Compatible with both iOS and Android platforms.
  • Ease of Use: Simple interface for integrating Gemma models into your Flutter projects.

Installation

Add flutter_gemma to your pubspec.yaml:

dependencies:
  flutter_gemma: latest_version

Run flutter pub get to install the package.

Setup

Download Model

  1. Obtain a pre-trained Gemma model (recommended: 2b or 2b-it) from Kaggle.
  2. Optionally, fine-tune the model for your specific use case.
  3. Rename the downloaded file to model.bin.

Integrate Model into Your App

iOS
  1. Enable file sharing in info.plist:
    UIFileSharingEnabled
    
  2. Change the linking type of pods to static by replacing use_frameworks! in Podfile with:
    use_frameworks! :linkage => :static
  3. Transfer model.bin to your device:
    • Connect your iPhone.
    • Open Finder, locate your iPhone under "Locations."
    • Click "Files" and drag model.bin to an app under the "Files" section, or use the "Add" button to upload model.bin.
Android
  1. Transfer model.bin to your device (for testing purposes):
    • Install the adb tool if not already installed.
    • Connect your Android device.
    • Copy model.bin to the output_path folder and push the contents to the Android device:
      adb shell rm -r /data/local/tmp/llm/ # Remove any previously loaded models
      adb shell mkdir -p /data/local/tmp/llm/
      adb push output_path /data/local/tmp/llm/model.bin
  2. For GPU support, add OpenCL support in AndroidManifest.xml above the </application> tag:
    
    
    
Web

Web currently supports only GPU backend models. Add dependencies to index.html in the web folder:

<script type="module">
import { FilesetResolver, LlmInference } from 'https://cdn.jsdelivr.net/npm/@mediapipe/tasks-genai';
window.FilesetResolver = FilesetResolver;
window.LlmInference = LlmInference;
</script>

Copy model.bin to your web folder.

Usage

Initialize the Plugin

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await FlutterGemmaPlugin.instance.init(
    maxTokens: 512,  // optional, default is 1024
    temperature: 1.0,  // optional, default is 1.0
    topK: 1,  // optional, default is 1
    randomSeed: 1,  // optional, default is 1
  );  
  runApp(const MyApp());
}

Generate a Response

final flutterGemma = FlutterGemmaPlugin.instance;
String response = await flutterGemma.getResponse(prompt: 'Tell me something interesting');
print(response);

Generate a Response as a Stream

final flutterGemma = FlutterGemmaPlugin.instance;
flutterGemma.getAsyncResponse(prompt: 'Tell me something interesting').listen((String? token) => print(token));

Generate a Chat Response

This method works properly only for instruction-tuned models:

final flutterGemma = FlutterGemmaPlugin.instance;
final messages = <Message>[];
messages.add(Message(text: 'Who are you?', isUser: true));
String response = await flutterGemma.getChatResponse(messages: messages);
print(response);
messages.add(Message(text: response));
messages.add(Message(text: 'Really?', isUser: true));
response = await flutterGemma.getChatResponse(messages: messages);
print(response);

Generate a Chat Response as a Stream

This method works properly only for instruction-tuned models:

final flutterGemma = FlutterGemmaPlugin.instance;
final messages = <Message>[];
messages.add(Message(text: 'Who are you?', isUser: true));
flutterGemma.getAsyncChatResponse(messages: messages).listen((String? token) => print(token));

Mini Project: Building a Simple Chatbot

Step 1: Setting Up the Project

  1. Create a new Flutter project.
  2. Add flutter_gemma to your pubspec.yaml.
  3. Initialize the FlutterGemmaPlugin in your main.dart.

Step 2: Creating the Chat Interface

import 'package:flutter/material.dart';
import 'package:flutter_gemma/flutter_gemma.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await FlutterGemmaPlugin.instance.init(
    maxTokens: 512,
    temperature: 1.0,
    topK: 1,
    randomSeed: 1,
  );  
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: ChatScreen(),
    );
  }
}

class ChatScreen extends StatefulWidget {
  @override
  _ChatScreenState createState() => _ChatScreenState();
}

class _ChatScreenState extends State<ChatScreen> {
  final _controller = TextEditingController();
  final List<String> _messages = [];
  final flutterGemma = FlutterGemmaPlugin.instance;

  void _sendMessage() async {
    final input = _controller.text;
    _controller.clear();
    setState(() {
      _messages.add('User: $input');
    });
    final response = await flutterGemma.getResponse(prompt: input);
    setState(() {
      _messages.add('Bot: $response');
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: Text('Chatbot')),
      body: Column(
        children: [
          Expanded(
            child: ListView.builder(
              itemCount: _messages.length,
              itemBuilder: (context, index) {
                return ListTile(title: Text(_messages[index]));
              },
            ),
          ),
          Padding(
            padding: const EdgeInsets.all(8.0),
            child: Row(
              children: [
                Expanded(
                  child: TextField(controller: _controller),
                ),
                IconButton(
                  icon: Icon(Icons.send),
                  onPressed: _sendMessage,
                ),
              ],
            ),
          ),
        ],
      ),
    );
  }
}

Step 3: Running the App

  1. Transfer the model.bin file to your device as described earlier.
  2. Run the app on your iOS or Android device.
  3. Start chatting with the bot!

Important Considerations

  • Models must be manually transferred to devices for testing. Network download functionality will be included in future versions.
  • Larger models (like 7b and 7b-it) may be too resource-intensive for on-device use.

Conclusion

With Flutter Gemma, integrating advanced AI capabilities into your Flutter applications has never been easier. By following this guide, you can set up the package, understand its usage, and create a simple chatbot to explore its potential. For more information and updates, keep an eye on the Flutter Gemma GitHub repository.

For more detailed tutorials on other Flutter libraries, check out our other blog posts:

Happy coding!

Leave a Reply

Your email address will not be published. Required fields are marked *