Why does random.sample() output differ between Kubernetes and local Docker?

It turns out that the difference in output is caused by how the Python random state is initialized (or rather, not advanced) between runs across different environments. In one environment (your local Docker container) the scheduled job is running in the same process (or thread), so the global random state is maintained and continuously advanced between job invocations. In the Kubernetes setup, however, the scheduled job is likely being run in a forked (or otherwise new) process that starts with the same inherited random state every time. This “reset” of the state makes random.sample yield the same sample on each run—unless you reseed it manually.

What’s Going On

• Single Process vs. Forked Processes:
  The Python random module seeds itself automatically (often via os.urandom if available) when the process starts. In a single process scenario (as appears to be happening locally), once seeded the generator’s state is continuously updated. With forking (or if processes are being recycled from a common parent), each child may inherit the same starting state if no additional seeding is performed.

• Apscheduler & Execution Environment:
  The AsyncIOScheduler runs within an event loop. On your local Docker container this loop is in a single process, whereas in Kubernetes it’s likely that the scheduling or executor setup (for example, via a pre-fork strategy or worker pool) leads to new processes being spawned that all start with the same state.

Recommended Fix

If you want the output to change with each scheduling even in a forked environment, you should “reseed” the random generator in each worker (or job). One common approach is to use a seed based on a high-resolution time or, even better, switch to a crypto-secure random generator (like random.SystemRandom) if you don’t need reproducibility.

For example, reseeding at the start of your function is one approach:

PYTHONCopied
import random
import time
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from pytz import utc

scheduler = AsyncIOScheduler(timezone=utc)
scheduler.start()

@scheduler.scheduled_job('interval', minutes=1, misfire_grace_time=None)
async def do_dada_news():
    # Reseed to ensure a different starting state in forked environments
    random.seed(time.time() * 10000)
    
    pages = [...]  # Your list (longer than 20 elements)
    print("---")
    for page in random.sample(pages, min(len(pages), 20)):
        print(page)

Alternatively, you may consider using an instance of SystemRandom:

PYTHONCopied
import random
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from pytz import utc

scheduler = AsyncIOScheduler(timezone=utc)
scheduler.start()

# Use SystemRandom which uses system sources for randomness
sys_random = random.SystemRandom()

@scheduler.scheduled_job('interval', minutes=1, misfire_grace_time=None)
async def do_dada_news():
    pages = [...]  # Your list
    print("---")
    for page in sys_random.sample(pages, min(len(pages), 20)):
        print(page)

This way you don’t have to worry about process forking affecting the state of the PRNG.

Summary

The discrepancy arises because in one setup the random state is continuously updated in a single process, while in another (due to forking or worker reuse in Kubernetes), each job invocation begins with the same inherited random state. Reseeding the generator (or using a system-based random instance) in your scheduled job is the proper fix to ensure different outputs each time.