Artificial Intelligence (AI) and Machine Learning (ML) require a lot of computing power. Specifically, you would want to have GPUs which have become the standard tool for computing-intensive applications because of their parallel processing capabilities, high throughput, and efficiency in handling the kinds of large-scale computations required for AI and ML.
Sometimes, I need to run a Large Language Model (LLM) but I found my laptop very slow for the size of model I needed. Other times, I need to train a model but again I found my laptop very slow for the architecture and the dataset I’m working with. But then I can’t really justify a dedicated machine with GPU(s) for AI/ML (or even gaming). So what to do?
Enter Vast.ai. It is a vast marketplace for renting GPU computing power, specifically designed for AI/ML workloads. It connects individuals or organizations that have underutilized GPUs with users who need affordable access to GPUs for tasks like AI/ML training, inference, or other high-performance computations. Your choices are wide (maybe too wide) and less expensive (but can be more expensive) than AWS or GCP offerings.
Renting GPUs with Vast.ai is pretty straightforward. You just have to have a good idea what you’re looking for.
Step 1: Create an Account on Vast.ai
1. Head over to Vast.ai: https://www.vast.ai
2. Click Console
3. Click Sign In. If you already have an account, Sign In. If not, Create A New Account
4. Complete your account setup. Make sure to verify your email.
5. On the Console, click Billing and Add Card. Add the the details of the credit card you will use for payment. Click Add Credit and add some credits to your account. Alternatively you can configure for automatic billing.
6. Click Account and Add SSH Key. Add your SSH public key you will use for securely accessing the rented server.
Step 2: Rent a GPU-Powered Server
Once you account has been setup and you’re logged in, it’s time to rent a server.
1. On the Console, click Search to search for a server with a GPU. You will have the following filters:
You can filter by #of GPUs, Rental type (interruptible, on-demand, reserved), GPU (Nvidia, AMD), Location, and other parameters (like Price).
You’ll want something with a GPU that can handle your planned workload. Something like the RTX 3090, the NVIDIA V100, or even A100, depending on your budget.
Be sure to pick a server that has enough RAM and GPU memory to run your tasks efficiently. At least 16GB RAM and 8GB GPU memory should be your baseline.
2. For template we’ll just use Ubuntu 22.04 VM for now. Click Change Template then search for “Ubuntu”. Select the template for the Ubuntu 22.04 VM template that shows up.
3. Once you’ve found a suitable server, click RENT.
4. Click Instances. You will see your instance CREATING…
Step 3: Access the Server
Once it is ready, you can now access your server.
1. Connect: Click >_ or CONNECT. You will be shown a dialog with the SSH command to use.
Copy and paste the command to your shell. It will look like this:
ssh -p <ssh port> root@<server IP> -L <local port>:localhost:<remote port>
The -L bit is important. This is for SSH tunneling. It allows you access your service as if your it’s running on your localhost without having to fiddle around with firewalls and ports on the remote server.
2. Install packages and dependencies:
Once logged in, you’ll want to make sure the server is updated
sudo apt update && sudo apt upgrade -yNow you can install packages and their dependencies.
And that’s it! Your Vast.ai GPU server is now ready to be configured for and run your AI/ML tasks.
Final Steps
Once you’re done with your tasks, it’s important to destroy the instance to save costs.
Backup your data: Don’t forget to back up any important results before shutting down the server.
Stop your instance: In Console, Instances, go to your running instance and click “Stop”. Note that you will still be charged for storage costs!
Destroy the instance: If you don’t need it anymore, you can terminate the instance to stop billing completely.