From idea to production with Python, Rust and GitLab CI
To search repositories and projects, login to gitlab.com . Suggestions GitLab Duo Agent Platform Code Suggestions (AI) CI/CD GitLab on AWS GitLab on Google Cloud Why GitLab? Published on: November 15, 2019 11 min read Bringing your application from idea to production using Python, Rust, and GitLab CI GitLab hero Mario Garcia demos the intricate process at GitLab Commit London. Sara Kassabian CI/CD community user stories During his talk at GitLab Commit London, GitLab Hero Mario García , explains how he troubleshooted his way through numerous roadblocks to take his Firebase application from development to production using Rust, Python and GitLab CI.
Rewriting from Python to Rust What is Rust? While Python is a household name among developers, Rust is the new kid on the block when it comes to a systems programming language. Rust was developed by Mozilla is giving to the world, it's been in development since 2009 with a first stable version released in May 2015 and it aims to improve memory usage while maintaining performance and speed. Mario, who is a Mozilla representative, dedicated himself to learning Rust in late 2015.
He started this journey by reading the Rust book, solving programming exercises , migrating Python code to Rust, and then rewriting one of his personal projects, a gallery for reveal.js presentations, in Rust . Reveal-js is a framework for creating presentations using HTML, and allows the user to store speaker notes, images, and more in a presentation gallery. Mario first wrote his gallery app in Python but migrated the project into Rust while he was learning the new language and found the process to be relatively painless.
But it wasn’t long before Mario hit a bump in the road when it came to using Rust for other projects. Problems with Rust “I was working on another project that I applied to the Mozilla Open Leaders program two years ago,” said Mario. “And for this project I was using Cairo SVG Python library . I needed this specific library because I was converting SVG files to PDF. So that's how I found out that it was impossible to rewrite this specific part with Rust because there is no alternative available in Rust for this library.” Not only did Rust lack an alternative to the CairoSVG Python library, but there was also no crates (Rust libraries) for Firebase.
Mario needed Firebase for his project that takes the database of speaker information and automatically generates certificates of participation. Mario was presenting an example of a web app at Google I/O Extended on how to use Rust and Firebase with web apps. But there was no functional library in Rust that could connect with Firebase and retrieve data from the database. Mario came up with a solution: use Python. More of a video person?
Watch Mario’s entire presentation from GitLab Commit London in the video below, or follow along step by step in this blog post. Using Python and Rust together In his presentation at GitLab Commit London, Mario demonstrated how he managed to build a Firebase web application in Rust using Python, and deploy it using GitLab CI so fellow GitLab users can try to replicate his process, or get some input if they're also having difficulties.
Configure your environment The first step is to make sure that your environment is properly configured. To use both Python, Rust, and GitLab CI, you’ll need the following on your machine: Git GCC Rust needs a C compiler and Cargo, which is the package manager for Rust projects Rust Nightly mode for this project Cargo Python 3.5+ pipenv for managing dependencies Install Rust using Rustup by typing the code below into your terminal. curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh You’ll also need to install bindings to run Python code directly from Rust, and that will also help with writing Python models using Rust code.
Mario recommends CPython and Py03 , but used CPython in this demo. Kick-start your project Next, Mario describes the general process for creating a project using Python and Rust. Cargo is a package manager for Rust projects, and will create a Cargo.toml file and src/ directory when its run. The Cargo.toml file is the manifest for the application and includes the dependencies the project requires. Within the src/ file is a main.rs file that contains an example of a Rust application.
The next step is to move through the src/ directory Cargo created to set up the default toolchain for the project. [package] name = "firebase_sample" version = "0.1.0" authors = [ "mattdark" ] edition = "2018" [dependencies] cpython = "0.3" serde = "1.0.99" serde_derive = "1.0.99" serde_json = "1.0.40" rocket = "0.4.2" [dependencies. rocket_contrib ] version = "0.4" features = [ "handlebars_templates" ] The Cargo.toml file will show the name of the application, the version, authors etc.
And if you’re working on Linux, it will take the user of your system and put it as the author of the project. “The dependencies that we need for the project are CPython for the Python part, Serde , which is a library that help us with reading information for files like JSON, and Rocket, which is a web framework for Rust,” said Mario. Next, set the Nightly version of Rust as the default toolchain for the project.
Add a ‘python’ directory to src/ directory, where you’ll be adding the Python models required for this project to this directory. Once the src/python is set-up, add the Pipfile or requirements.txt file for the dependencies of the Python module to the directory. [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [dev - packages] [packages] firebase = "*" python - jwt = "*" gcloud = "*" sseclient = "*" pycrypto = "*" requests - toolbelt = "*" [requires] python_version = "3.7.3" The Pipfile is an example of a project used for Firebase.
Included here is all the dependencies we need for Firebase in the file, as well as the Python version in use. Next write the Rust code in src/main.rs and add the Python scripts in src/python. Writing the Python code Mario’s Firebase application is designed to rake a database of speaker information and automatically generate certificates of participation in PDF format. { "slides" : { "privacymatters" : { "description" : "Talk about privacy & security" , "file" : "privacy-matters.md" , "id" : "2" , "screenshot" : "/img/screenshot/privacy-matters.png" , "theme" : "mozilla.css" , "title" : "Why Privacy Matters?" , "url" : "privacy-matters" }, "rust101" : { "description" : "Introduction to Rust" , "file" : "rust-101.md" , "id" : "1" , "screenshot" : "/img/screenshot/rust-101.png" , "theme" : "mozilla.css" , "title" : "Rust 101" , "url" : "rust-101" }, "rustrocket" : { "description" : "Building Web Apps with Rust + Rocket" , "file" : "rust-rocket.md" , "id" : "3" , "screenshot" : "/img/screenshot/rust-rocket.png" , "theme" : "mozilla.css" , "title" : "Rust + Rocket" , "url" : "rust-rocket" }, "whyrust" : { "description" : "What is Rust and Why Learn it?" , "file" : "why-rust.md" , "id" : "4" , "screenshot" : "/img/screenshot/why-rust.png" , "theme" : "mozilla.css" , "title" : "Why Rust?" , "url" : "why-rust" } } } Information about Mario’s Firebase application lives in this JSON file of the Firebase database.
The application is written in Rust, and therefore needed a Firebase connector. But since the is not a functional Firebase crate, Mario had to think outside the box and use the Python library. import json from firebase import Firebase def read_data(self): config = { "apiKey": "APIKEY", "authDomain": "fir-speakers.firebaseapp.com", "databaseURL": "https://fir-speakers.firebaseio.com", "projectId": "fir-speakers", "storageBucket": "", "messagingSenderId": "MESSAGINGSENDERID" } firebase = Firebase(config) speaker = list() db = firebase.database() all_speakers = db.child("speakers").get() for x in all_speakers.each(): speaker.append(x.val()) s = json.dumps(speaker) return s “For the Python part of the project, we have to connect
The rest of this page is on the live site.