Enhancing App Development: Integrating GitHub Models with Firebase

Introduction of Shabbi and Open Source Friday

• Welcome to Open Source Friday, a GitHub power sponsor Community-led event that highlights projects, maintainers, contributors, and the ecosystem, and also explores new projects, with today's guest being Shabbi, a super special guest who will present in English today and in Spanish next week (2m14s).
• The host is back in their home office after a bit of forced travel due to hurricanes and weather situations, and is sending love to those affected, and is happy to be back for another edition of Open Source Friday (2m21s).
• Open Source Friday is an event that highlights not only projects but also the amazing maintainers, contributors, and the ecosystem that everyone depends on, and gives an opportunity to explore new projects (2m55s).
• Shabbi is a guest who was met at the GitHub Universe event last year, and is a nice person, open source contributor, maintainer, and creator, making it a great pleasure to have him on the show (4m30s).

Shabbi's Background and Achievements

• Shabbi was born in Valencia, currently lives in Madrid, and has over 7 years of experience in cloud, Kubernetes, DevOps, QA, microservices, conversational AI, and mentoring startups (4m54s).
• Shabbi is part of the Google for Startups program, an organizer of GDG Madrid, Deast Madrid, and the Alexa community in Spanish, and enjoys music, friends, and is the author of a book called The Alexa Revolution (5m17s).
• Shabbi is a Microsoft MVP, GitHub Star, Alexa Champion, Google Developer Expert, Google Cloud Champion Innovator, Machine Learning AWS Community Builder, and a Content Hero (5m37s).

GenKits Project Overview

• The project that Shabbi will be presenting today is called GenKits (6m3s).
• An open-source plugin has been created to integrate GitHub models with Firebase, allowing users to access GitHub models and incorporate them into their applications (6m6s).
• The plugin's creator, Shabi, was inspired to develop the plugin after gaining early access to GitHub models as a GitHub star and being impressed by their ease of use (7m46s).
• Shabi was also influenced by the Yen Kit framework from the Firebase team, which allows for the creation of generative AI apps, but noticed the lack of a GitHub models plugin and decided to build one (8m46s).
• The plugin is fully functional and enables users to create generative AI applications using GitHub models in under five minutes (9m34s).

GenKits Plugin Ecosystem and Development

• The plugin is part of the Yen Kit plugin ecosystem, which allows developers to create their own plugins (9m3s).
• Shabi is sharing their screen to demonstrate the plugin's capabilities and show the available GitHub models (7m58s).
• The plugin's development was rapid, with Shabi starting work on the idea shortly after GitHub models were announced (10m12s).

Firebase Gen Kit Framework

• Firebase Gen Kit is an open-source project and a framework that allows users to create generative AI applications in a super easy way, with a focus on ease of use and developer experience (10m42s).
• Unlike other frameworks like Llama Index and Lang Chain, Firebase Gen Kit is designed to be straightforward and easy to use, without requiring users to learn a complex abstraction layer (11m32s).
• Firebase Gen Kit is not limited to Firebase functions and can be used as a couple of libraries to deploy whatever users want, with support for Node, JavaScript, TypeScript, and Go (in alpha) (12m37s).
• The framework has a plugin ecosystem that allows users to create different types of plugins, connect to different models, and use various LLMs and embedding models (13m19s).
• Firebase Gen Kit has official plugins for Google Cloud, Firebase, and vector databases, and supports building retrieval-augmented generation applications (14m19s).
• The framework also features tool calling, which allows users to define external functions that LLMs can call, covering most of the needs for building a generative AI app (15m16s).

GitHub Models and Integration

• GitHub Models is a marketplace where users can find available models, accessible by going to github.com/marketplace and clicking on models (15m51s).
• GitHub models are easy to use, requiring only a GitHub account to access and interact with models like GPT 40, which can be tested directly in a playground or through an API (15m59s).
• There are two ways to interact with the API: using the vendor SDK (e.g., Open AI SDK) or the Aure AI Inference SDK, with the latter recommended for building generic applications that can use different models (16m50s).
• The Aure AI Inference SDK requires only a GitHub token for login and allows for easy switching between models by changing an environment variable (17m47s).
• This makes it simple to test and compare different models, such as GPT 40 mini or Co:here, without requiring significant changes to the code (18m8s).
• Previously, integrating different models into an application would require writing specific code for each model, but the Aure AI Inference SDK simplifies this process (18m41s).
• The ability to easily switch between models makes it more accessible to play with and compare different models to determine which one is best for a specific application or use case (19m19s).
• The Aure AI Inference SDK also allows for easy AB testing and integration of new models, such as GPT 5, by simply changing an environment variable (20m4s).
• This feature is particularly useful for testing and comparing different models, as new models are being released frequently (20m10s).

Using GenKit with GitHub Models

• Genen kit is recommended for use with GitHub models due to its numerous plugins and integrations that make the process easier (20m48s).
• To use the Gen kit GitHub models plugin, one needs to install the Gen kit CLI and set up a Fireb Gen kit plugin, with instructions available in the Gen kit documentation (21m40s).
• After setting up the plugin, users can install the Jenex GitHub plugin in their JavaScript or TypeScript project to use GitHub models (22m14s).
• Genen kit works by configuring the framework and specifying the plugins to be used, with the GitHub plugin initialized by setting the GitHub token environment variable or specifying a line of code (22m37s).
• To use a GitHub model in Genen kit, users can call the generate function with a prompt and receive a response, as demonstrated in a simple example application (23m6s).
• The Gen kit GitHub models plugin allows for easy interaction with GitHub models, including the use of the generate function, tools, and images as input (24m59s).
• The plugin's capabilities include accepting structor input and output, enabling users to call large language models (LLMs) and retrieve information from them (24m36s).

Example Use Cases and Capabilities

• Example use cases for the plugin include translation, such as translating an image from English to Spanish by specifying the input and calling the GitHub model (25m30s).
• The GitHub model supports JSON output, structor output, and models for retrieval and generation in Genen kit, which is also available for creating embeddings (25m43s).

Awesome Fireb Kit Repository

• There is a repository called "Awesome Fireb Kit" that contains a list of relevant information about Firebase, including plugins for Fire Genen kit, which are official, supported by Google, and created by the community (26m31s).
• The Awesome Fireb Kit repository includes various plugins such as model embedding plugins, vector store plugins, monitoring plugins, and deployment plugins for Firebase functions (27m14s).
• The repository also includes plugins for the JavaScript community, such as Mine the Gen kit GitHub, Anthropy, Coher, Gro Mistral, Open AI, and Aure Open AI (27m52s).
• There are also plugins for vector store databases, including Convex Engine, NSW Quadrant, Astra PG, Vector PB, and Redis (28m5s).
• The repository contains evaluator plugins and other plugins, as well as official plugins for Go, such as Verx AI and AMA (28m26s).
• The AMA plugin allows users to run AMA on their laptop and use models like Gemma and Gemma CH billion without consuming tokens in the cloud (28m47s).
• The Awesome Fireb Kit repository includes talks, videos, articles, tutorials, and contributions, which are open to the community (29m14s).
• The repository is hosted on GitHub under the username "shabob" and is open for contributions and additions (29m35s).
• The repository is a crowdsourced list of plugins and resources, and users are encouraged to contribute and add new plugins and resources (30m1s).
• The repository emphasizes that Firebase is not the only option, and users can deploy their code wherever they want (30m24s).

GenKit and Goan Community

• GenKit is a framework for Firebase serverless functions that is simpler and faster to iterate than other frameworks like LineChain or Lama Index, making it easier to start experimenting with models (30m48s).
• The Goan community is happy about GenKit's support for Goan, a language that offers good performance and flexibility (31m21s).

NLU and Jenit NLU Project

• Natural Language Understanding (NLU) is a technology that identifies users' actions, such as understanding that a user wants to buy shoes from a specific brand like Nike (32m46s).
• NLU is used in chatbots to detect what users want and plan accordingly, and it is also used in Large Language Models (LLMs) (33m28s).
• A project called Jenit NLU was created using GitHub models and GenKit, which is a serverless project that can be easily deployed to Firebase (34m3s).
• The project has two parts: a Firebase function project and a function itself, which contains the source code in the index.js file (34m33s).
• To use GenKit, it is necessary to call the configure function to enable or add plugins, such as the GitHub models and the dot prompt engine (35m1s).
• The dot prompt engine is used to manage prompts, and it is one of the features that makes GenKit stand out (35m20s).
• OnFlow is a concept in Jenkit that allows for complex interactions by calling one flow from another, enabling the creation of more intricate processes (35m33s).
• To create a flow, a name must be given, and the input and output schemas must be specified, with the input schema defining the structure of the data being passed into the flow (36m5s).
• In this case, the input schema is an object type with a property called "text" that is a string, and the output schema is just a string (36m28s).

NLU Model and Intents

• The flow's main function is to define a schema, and before diving into the code, it's essential to understand the NLU (Natural Language Understanding) model (37m1s).
• The NLU model has intents and entities, which are used to extract or detect what users are trying to communicate (37m9s).
• In this example, there is an intent called "order shoes" with different utterances, such as "I want a pair of shoes from S brand" (37m28s).
• The entity "S brand" is defined in the entities JAML file, which can include multiple examples, such as "Puma", "Nike", and "Adidas" (37m59s).
• One of the benefits of using GitHub models and LLMs (Large Language Models) is that they can abstract and understand variations of examples, even if they are not explicitly listed (38m51s).
• The goal is to extract the ID of the intent and the value of the "S brand" entity, which can be sent to the backend (39m47s).

Output Schema and Code Implementation

• To achieve this, the output schema must be defined, which includes the intent and the list of recognized entities (40m21s).
• The code defines a list of shoes detected, which is used as an example, and calls a prompt named "nlu" that is stored under the "prompts" folder, with the actual prompt files being separate from the TypeScript code for abstraction purposes (40m48s).
• The "nlu" prompt is defined with a specific structure, including the model used (GPT-40), input schema (intents, entities, and user input), and output schema (NLU output in JSON format) (41m45s).
• The input schema includes a database of intents and entities, which are defined in JAML files, and the user's request, while the output schema is defined as NLU output (41m50s).
• The prompt is designed to detect intents and extract entities from a given text, with placeholders for intents, entities, and user input (42m39s).
• The code reads the JAML databases for intents and entities, which can be stored externally using Firebase or another storage solution, and calls the "generate" function with the intents, entities, and user request as inputs (43m18s).
• The result is an output in JSON format, which can be used for further processing (43m59s).

Genie Kit Developer UI and GitHub Models

• The Genie Kit has a developer UI that allows for testing and interaction with the models, including the ability to run the Firebase emulator and the Genie Kit UI locally (44m13s).
• The Genie Kit supports 23 GitHub models, which can be interacted with from the developer UI, allowing for testing and experimentation with different models (45m28s).
• Firest Jenit is a developer UI that allows users to interact with GitHub models in one place, eliminating the need to go to the GitHub Marketplace and choose a model separately (45m46s).
• The UI can be used to test models, such as the GPT 4 model, by providing input and receiving output, which includes intent, order, and detected entities (46m4s).
• The input schema is an object with a property called "text" that is a string, allowing users to type whatever they want and click on "run" to get the output (46m43s).
• The model can abstract and understand input in different languages, such as Spanish, and provide the same output as if the input was in English (47m44s).
• The model can also handle entities that are not in the database, such as a different brand name, and provide the correct output (47m11s).
• The example shown is a basic chatbot for an e-commerce site, but the possibilities are endless, and the lines of code required are less than 50 (49m15s).
• The combination of Firest Jenit and GitHub models makes it easy to create apps, and the limit is only in the user's imagination (49m56s).

Firest Jenit Use Cases and Open Source Code

• There are already use cases for Firest Jenit, including Enterprise applications, despite the rate limits being low due to the beta status of the Giam models (50m41s).
• The code showcased is open source and can be found on the GitHub account "shabob" under the repository "jeny" and "shabobfarestYenkit", where users can access the source code, understand the usage, and explore the database of intents and entities (51m0s).
• The GitHub repository is open to contributions, and users are encouraged to fork, contribute, or open issues to help improve the project (51m40s).
• The creator can be found on GitHub at "shabop" and on Twitter at "savid", where they are active and open to feedback and discussions (52m10s).

Project Contributions and Future Enhancements

• The current GitHub models do not support audio or image generation, but the plugin only supports image and text input, and users are encouraged to contribute to adding audio support in the future (53m5s).
• The creator is open to feedback and encourages users to open issues or pull requests to contribute to the project, with a focus on adding new features and improving the plugin (53m52s).

Project Popularity and GitHub Universe

• The plugin has gained popularity, with around 128 downloads per week, and the creator is excited for the future of the project and the upcoming GitHub Universe event (54m16s).
• The creator will be attending the GitHub Universe event in San Francisco and will be hosting a session in Spanish, encouraging Spanish speakers to participate and ask questions (55m4s).
• The GitHub Universe event will be celebrating its 10th year anniversary, and the creator is excited for the new features and announcements that will be made during the event (55m45s).
• The speaker is open to invitations and will be giving a workshop in Glasgow at the end of next month, and also plans to visit Spain, specifically Madrid and Valencia, and is open to other invitations (56m32s).

Security and Contributing Guidelines

• For security purposes, the recommendation is to use tools like SonarCloud to check for vulnerabilities in every commit and set up a quality gate for every pull request (57m40s).
• GitHub has a dependency bot that can be enabled to check for security vulnerabilities in dependencies, and this feature is totally free (58m2s).
• To contribute to the project, one should go to the repository, read the contributing guidelines, and look for impactful contributions, such as creating real applications using the project's features (58m43s).
• The most valuable contributions would be to create real-world applications using the project's features, which would be fulfilling to see the project being used in real-life scenarios (59m1s).
• The project can be integrated into various web apps, including e-commerce, and there are many use cases for the project, but the speaker is looking for real-world applications using the project's features (59m38s).

GitHub Models Use Cases

• GitHub models can be integrated with a chatbot to provide a knowledge base for businesses, allowing users to ask questions and receive answers, and can also be used to summarize content, such as books or articles, by providing a brief summary of the main points (1h0m0s).
• Another use case for GitHub models is in e-learning, where a web app can be built to create an entire course with chapters, challenges, and quizzes based on user input, making it easier for creators and teachers to produce content (1h0m46s).
• GitHub models can also be used to create personalized quizzes for learning, which can help retain more information by constantly stopping and quizzing, and can be applied to various subjects and topics (1h2m13s).
• Additionally, GitHub models can be used for image-to-text applications, such as object detection, and can be integrated with other tools and technologies to build more complex applications (1h2m41s).
• A specific example of an app built using GitHub models is a baby food analyzer, which can detect the ingredients in a picture of a meal and provide suggestions for preparation based on the baby's age and allergies (1h2m52s).
• The baby food analyzer app uses a large language model (LLM) to detect the ingredients and provide suggestions, but it is essential to note that the app should not be relied upon solely for making decisions about a baby's diet (1h3m32s).
• A potential app idea using LLMs is a baby food advisor that detects ingredients like chocolate or sugar and provides warnings about their suitability for early babies, utilizing models like GBT-40 or ITAM model (1h4m17s).

GitHub Universe and Real-World Problem Solving

• There's a meme about using AI to solve real problems that take time and effort, rather than just creating art or writing books, which is a cool concept (1h4m52s).
• GitHub Universe is an event that offers a great experience, featuring talks, keynotes, new releases, and learning opportunities from real use cases and GitHub experts (1h5m58s).
• The event also provides excellent networking opportunities, allowing attendees to meet maintainers, open-source contributors, and important figures from the open-source community (1h6m24s).
• GitHub Universe offers various activities, including a lunch for certifications, a GitHub store with merchandise, and a chance to get a GitHub certificate by taking an exam for free (1h6m57s).
• The event is a great place to interact with people, including the creators of popular projects, who are often lovely and accessible (1h7m54s).
• The "hallway track" at GitHub Universe allows attendees to stop and talk to people they admire, creating opportunities for meaningful connections and relationships (1h7m47s).
• The event's lineup of speakers is epic, and the overall experience is amazing, making it an event worth attending (1h7m41s).
• GitHub Universe is celebrating its 10th year anniversary and attendees are encouraged to say hello and provide feedback to the team, with the event being for the attendees' benefit (1h8m16s).

Hosting Options and Giveaway Winner

• The difference between hosting with Firebase and GitHub Pages depends on the use case, and it is recommended to test both options to determine which one works best (1h9m21s).
• The speaker emphasizes the importance of using the framework that makes one work the happiest, fastest, and most productive (1h9m55s).
• A giveaway was held for a ticket to GitHub Universe, and the winner, Brian Shand, will receive an email with details on how to register and is invited to meet and be interviewed on the podcast (1h11m20s).
• The winner will also get to attend GitHub Universe in person and have a donut with the speaker (1h10m55s).
• The speaker thanks everyone for participating in the giveaway and congratulates the winner (1h11m10s).

Upcoming Events and Announcements

• An upcoming event called Open Source Bees will be held next Friday, where Shabi will demo Jit and his plugin in Spanish, catering to Spanish-speaking communities (1h12m7s).
• The speaker has one more announcement to share, which is about an event in Johannesburg, South Africa, where GitHub is hosting a community event (1h12m45s).
• The event in Johannesburg will feature a keynote by GitHub's CEO, a panel discussion, and participation from local partners, including Allin for Open Source Africa and Open Source Community Africa (1h13m30s).
• The event will also include a hackathon and will be an intimate gathering, with registration available on GitHub Constellation's website (1h14m5s).
• The event is scheduled to take place on November 14, and attendees are encouraged to sign up early, with the possibility of meeting the speaker and other community members in person (1h14m7s).

Closing Remarks and Johannesburg Event

• The speaker thanks the audience for their support of open source and congratulates the winner of an unspecified competition, with plans to send a message shortly (1h14m35s).
• The speaker invites viewers to leave questions and comments, which will be forwarded to Shabi for response, and reminds everyone to save the date for the Johannesburg event (1h14m47s).
• The event promises to be an epic day with swag, awesome sessions, and possibly donuts, with the speaker expressing excitement about meeting the community in person (1h15m0s).

Conference Experience and Atmosphere

• The conference has a super welcoming atmosphere and a community feel, which is amazing and makes the speaker feel grateful for the opportunity to speak there (1h16m7s).
• The speaker delivered their session and is looking forward to exploring the new sessions and learning from them (1h16m24s).
• The speaker is impressed by the coaching and organization of the conference (1h16m34s).
• The speaker has spoken at many conferences, but this one is different due to its non-corporate driven and smaller size, making it feel less loud and more approachable (1h16m43s).
• The speaker has made friends with many people at the conference and feels approachable themselves, enjoying the experience so far (1h16m50s).