I created this to test my streaming pipeline that stream files from MongoDB
before use:
- rename file
.env.exampleto.envand add to it your connection configs - run
pip install -r requirements.txt - you can edit
upload.pyandstream.pyto add your configs
Warning
you have to configure replica set in MongoDB
python upload.pya new folders source/ & archive/ will be created (if not exist)
Now add files to source/ folder and watch them upload
uploaded files will move from source/ to archive/
python stream.py