How to Run Python Scripts 24/7 on a Free VPS (Web Scraping & Automation)

Running Python scripts on your laptop means they stop when you close the lid. Running them on a free VPS means they run forever — 24/7, even when your computer is off, your internet is down, or you’re asleep.

This guide covers everything from setting up Python on a fresh VPS to keeping scripts alive permanently with systemd, scheduling them with cron, and running web scrapers with BeautifulSoup and Scrapy. All on a free VPS — no credit card required. Before diving in, make sure you know the 5 things you must do after launching your Linux VPS to set up a secure base.

Why a VPS Beats Every Other Option for Python

Here’s the honest breakdown of where developers try to run Python scripts 24/7 and what actually happens:

A VPS is a real server. Your scripts run as Linux services — they start on boot, restart on crash, and run indefinitely with no time limits or resource throttling on your process.

Step 1 — Get a Free VPS

You need a VPS with a public IP, SSH access, and enough resources to run your scripts. VPSWala’s free VPS gives you a real Ubuntu server with root access, no credit card required. For Python web scraping and automation, the ARM64 Starter plan handles lightweight scripts, while the 30-day Professional free trial (8-core AMD EPYC, 8 GB RAM) handles heavy concurrent scrapers.

Connect via SSH once deployed. New to SSH? Our SSH tips and tricks guide for managing a VPS covers everything you need to know:

ssh root@YOUR_VPS_IP

Step 2 — Set Up Python and Virtual Environment

Ubuntu 22.04 and 24.04 both ship with Python 3. Check the version:

python3 --version

Install pip and venv:

sudo apt update
sudo apt install -y python3-pip python3-venv python3-dev build-essential

Need the very latest Python? Our guide on installing Python 3.13 on Ubuntu covers building from source to get the newest version.

Create a project directory and virtual environment:

mkdir ~/myproject
cd ~/myproject
python3 -m venv venv
source venv/bin/activate

Your prompt will change to show (venv) — you’re now inside the virtual environment. Install your dependencies:

# Example for a web scraping project
pip install requests beautifulsoup4 lxml scrapy
 
# Example for a data/automation project
pip install pandas numpy schedule
 
# Save dependencies to requirements.txt
pip freeze > requirements.txt

To deactivate the virtual environment when you’re done working: deactivate

Step 3 — Upload Your Python Script

Three ways to get your script onto the VPS:

Option A — Git (recommended)

cd ~
git clone https://github.com/YOUR_USERNAME/YOUR_REPO.git myproject
cd myproject
source venv/bin/activate
pip install -r requirements.txt

Option B — SCP from local machine

# Run on your LOCAL machine
scp /path/to/script.py root@YOUR_VPS_IP:~/myproject/

Option C — Create directly on the server

nano ~/myproject/script.py

Paste your code, save with Ctrl+O, exit with Ctrl+X.

Step 4 — Run Script in Background with nohup

The simplest way to keep a script running after you close your SSH session:

cd ~/myproject
source venv/bin/activate
nohup python3 script.py > output.log 2>&1 &

Breaking this down:

• nohup — ignores the hangup signal when you disconnect
• > output.log — redirects stdout to a log file
• 2>&1 — redirects stderr (errors) to the same log file
• & — runs the process in the background

Check if it’s running:

ps aux | grep script.py

View the output:

tail -f output.log

Kill the process if needed:

pkill -f script.py

Limitation of nohup: It doesn’t auto-restart if the script crashes, and it doesn’t survive a server reboot. Use systemd for production scripts.

Step 5 — Run as a systemd Service (Best Method)

systemd is the right tool for scripts that need to run continuously — it auto-restarts on crash, starts on boot, and integrates with system logging. This is how professionals run Python in production.

Create a service file:

sudo nano /etc/systemd/system/myscript.service

Paste this — replace paths and username with your own:

[Unit]
Description=My Python Script
After=network.target
Wants=network-online.target
 
[Service]
Type=simple
User=root
WorkingDirectory=/root/myproject
ExecStart=/root/myproject/venv/bin/python3 /root/myproject/script.py
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
 
[Install]
WantedBy=multi-user.target

Key settings explained:

• After=network.target — script only starts once the network is up (critical for scrapers)
• Restart=always — restarts automatically if the script crashes or exits
• RestartSec=10 — waits 10 seconds before restarting, prevents rapid crash loops
• ExecStart — uses the Python interpreter inside your venv, so all installed packages are available

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable myscript.service
sudo systemctl start myscript.service

Check the status:

sudo systemctl status myscript.service

You should see active (running). Your script now starts automatically on every boot and restarts within 10 seconds of any crash.

Step 6 — Schedule with Cron (for Periodic Scripts)

If your script doesn’t run continuously but needs to run at specific times — for example a scraper that runs every hour or a price monitor that runs at midnight — use cron instead of systemd.

crontab -e

Add a line at the bottom. Cron format: minute hour day month weekday command

# Run every hour
0 * * * * /root/myproject/venv/bin/python3 /root/myproject/script.py >> /root/myproject/cron.log 2>&1
 
# Run every day at 2 AM
0 2 * * * /root/myproject/venv/bin/python3 /root/myproject/script.py >> /root/myproject/cron.log 2>&1
 
# Run every 15 minutes
*/15 * * * * /root/myproject/venv/bin/python3 /root/myproject/script.py >> /root/myproject/cron.log 2>&1
 
# Run every Monday at 9 AM
0 9 * * 1 /root/myproject/venv/bin/python3 /root/myproject/script.py >> /root/myproject/cron.log 2>&1

Always use the full path to the Python interpreter inside your venv — cron doesn’t load your shell environment.

Step 7 — Web Scraping with BeautifulSoup and Scrapy

Basic scraper with BeautifulSoup

This scrapes a webpage and saves the data to a file:

pip install requests beautifulsoup4

# scraper.py
import requests
from bs4 import BeautifulSoup
import json
from datetime import datetime
 
def scrape_page(url):
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
    }
    response = requests.get(url, headers=headers, timeout=10)
    response.raise_for_status()
 
    soup = BeautifulSoup(response.text, 'lxml')
 
    # Example: scrape all article titles from a news site
    titles = []
    for article in soup.find_all('h2', class_='article-title'):
        titles.append({
            'title': article.get_text(strip=True),
            'timestamp': datetime.now().isoformat()
        })
 
    return titles
 
if __name__ == '__main__':
    url = 'https://example.com/news'
    data = scrape_page(url)
 
    with open('output.json', 'w') as f:
        json.dump(data, f, indent=2)
 
    print(f"Scraped {len(data)} items at {datetime.now()}")

Running Scrapy spiders on a schedule

Scrapy is better for large-scale scraping with pagination, concurrency, and pipelines:

pip install scrapy
scrapy startproject myspider
cd myspider
scrapy genspider example example.com

Run your spider from cron or systemd:

# In crontab — runs the spider every 6 hours
0 */6 * * * cd /root/myproject/myspider && /root/myproject/venv/bin/scrapy crawl example -o output.json >> /root/scraper.log 2>&1

Rotate user agents to avoid blocking

pip install fake-useragent

from fake_useragent import UserAgent
 
ua = UserAgent()
headers = {'User-Agent': ua.random}
response = requests.get(url, headers=headers)

Add delays to be polite

import time
import random
 
# Random delay between 2 and 5 seconds between requests
time.sleep(random.uniform(2, 5))

Step 8 — Headless Browser Scraping with Selenium

For sites that require JavaScript to render content, use Selenium with a headless Chrome browser:

sudo apt install -y chromium-browser chromium-chromedriver
pip install selenium

# selenium_scraper.py
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
import time
 
def get_driver():
    options = Options()
    options.add_argument('--headless')           # No GUI
    options.add_argument('--no-sandbox')         # Required on VPS
    options.add_argument('--disable-dev-shm-usage')  # Required on VPS
    options.add_argument('--disable-gpu')
    options.add_argument('--window-size=1920,1080')
 
    driver = webdriver.Chrome(options=options)
    return driver
 
driver = get_driver()
driver.get('https://example.com')
time.sleep(3)  # Wait for JS to render
 
# Find elements
items = driver.find_elements(By.CLASS_NAME, 'product-name')
for item in items:
    print(item.text)
 
driver.quit()

--no-sandbox and --disable-dev-shm-usage are required on VPS environments — without them Chrome will fail silently.

Step 9 — Monitor and View Logs

For systemd services, logs go to journald by default:

# View live logs
sudo journalctl -u myscript.service -f
 
# View last 100 lines
sudo journalctl -u myscript.service -n 100
 
# View logs since today
sudo journalctl -u myscript.service --since today
 
# Check service status
sudo systemctl status myscript.service
 
# Restart the service
sudo systemctl restart myscript.service

For nohup or cron scripts, view the log file directly:

tail -f ~/myproject/output.log

To add structured logging to your Python script:

import logging
 
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('/root/myproject/script.log'),
        logging.StreamHandler()
    ]
)
 
logger = logging.getLogger(__name__)
logger.info("Script started")
logger.error("Something went wrong")

Python VPS Use Cases

Troubleshooting

Your Python script is now running 24/7 on a server you don’t have to babysit. systemd handles crashes, cron handles schedules, and your VPS handles everything else while you sleep.

For scraping-heavy workloads, multiple concurrent scrapers, or scripts that process large datasets, the VPSWala Professional free trial gives you 8-core AMD EPYC and 8 GB DDR5 ECC RAM — more than enough to run 10+ scrapers simultaneously without the memory constraints you’d hit on a basic free tier.

Next steps: learn how to monitor system resources on your Linux VPS to track CPU and RAM usage as your scripts run, and set up Docker containers on your VPS if you want to run scripts in isolated environments. If you need a Windows VPS to run .exe-based automation tools, grab a free RDP server here.