mirror of
https://github.com/Bostame/onboarding_system.git
synced 2025-12-06 07:55:27 +01:00
Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| eb08b565a0 | |||
| 1053eb898d |
188
README.md
188
README.md
@@ -1,188 +0,0 @@
|
|||||||
|
|
||||||
# onboarding_system
|
|
||||||
|
|
||||||
This project automates the synchronization of local folders (`attachments` and `onboarded_person`) with corresponding directories in a Nextcloud instance. It checks for new or modified files in local folders and uploads them to the designated Nextcloud directories, using a persistent session for efficient authentication. Additionally, it includes features for email notification and PDF generation from HTML templates and CSV files.
|
|
||||||
|
|
||||||
## Features
|
|
||||||
- Generate pdf from the dynamic CSV file.
|
|
||||||
- Send email to to the recipient when it is necessary.
|
|
||||||
- Automatically syncs files between local folders and Nextcloud.
|
|
||||||
- Uploads only new or modified files to Nextcloud.
|
|
||||||
- Processes CSV attachments from emails to generate PDFs.
|
|
||||||
- Sends email notifications based on specific CSV content.
|
|
||||||
- Configurable synchronization interval (default is 60 seconds).
|
|
||||||
- Uses a persistent HTTP session for efficient communication with Nextcloud.
|
|
||||||
- Tracks files locally to prevent re-uploading of unchanged files.
|
|
||||||
|
|
||||||
## Requirements
|
|
||||||
|
|
||||||
- Python 3.8+
|
|
||||||
- Required Python libraries:
|
|
||||||
- `requests`
|
|
||||||
- `python-dotenv`
|
|
||||||
- `HTTPBasicAuth` (from `requests`)
|
|
||||||
- `pandas`
|
|
||||||
- `jinja2`
|
|
||||||
- `pdfkit`
|
|
||||||
- `PyPDF2`
|
|
||||||
- `imaplib`
|
|
||||||
- `email`
|
|
||||||
- `smtplib`
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
### Step 1: Clone the Repository
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone https://github.com/Bostame/onboarding_system.git
|
|
||||||
cd onboarding_system
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: Install Dependencies
|
|
||||||
|
|
||||||
Make sure you have Python 3.8+ installed. Use the following command to install dependencies:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -r requirements.yml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3: Configure Environment Variables
|
|
||||||
|
|
||||||
Create an `.env` file inside the `env/` directory with the following content:
|
|
||||||
|
|
||||||
```env
|
|
||||||
# Nextcloud credentials and URLs
|
|
||||||
NEXTCLOUD_BASE_URL=https://your-nextcloud-url/remote.php/dav/files/your-user
|
|
||||||
NEXTCLOUD_USERNAME=your-username
|
|
||||||
NEXTCLOUD_PASSWORD=your-password
|
|
||||||
NEXTCLOUD_DIRECTORY=Group-on-off-boarding
|
|
||||||
|
|
||||||
# Sync interval in seconds (default: 60 seconds)
|
|
||||||
SYNC_INTERVAL=60
|
|
||||||
|
|
||||||
# Local folder paths
|
|
||||||
TEMPLATES_DIR=templates
|
|
||||||
ATTACHMENTS_DIR=attachments
|
|
||||||
ONBOARDED_DIR=onboarded_person
|
|
||||||
TEMP_PDF_DIR=temp_pdf
|
|
||||||
EMAIL_TEXT_DIR=email_text
|
|
||||||
|
|
||||||
# Email details (if applicable)
|
|
||||||
IMAP_SERVER=imap.example.com
|
|
||||||
SMTP_SERVER=smtp.example.com
|
|
||||||
EMAIL_PORT=465
|
|
||||||
EMAIL_ACCOUNT=your-email@example.com
|
|
||||||
PASSWORD=your-email-password
|
|
||||||
MINUTH_EMAIL=minuth@example.com
|
|
||||||
DRITICH_EMAIL=dritich@example.com
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 4: Create Local Folders
|
|
||||||
|
|
||||||
Ensure the following local directories exist:
|
|
||||||
|
|
||||||
- `attachments/`
|
|
||||||
- `onboarded_person/`
|
|
||||||
- `templates/`
|
|
||||||
- `temp_pdf/`
|
|
||||||
- `email_text/`
|
|
||||||
|
|
||||||
You can modify these folder paths via the `.env` file as needed.
|
|
||||||
|
|
||||||
### Step 5: Run the Script
|
|
||||||
|
|
||||||
To manually run the synchronization and CSV processing:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python sync_script.py
|
|
||||||
python main.py
|
|
||||||
```
|
|
||||||
|
|
||||||
### Creating Systemd Service Units
|
|
||||||
|
|
||||||
To continuously run the synchronization and processing in the background, create systemd service units.
|
|
||||||
|
|
||||||
#### Step 1: Create the `onboarding-system.service` and `nextcloud-sync.service` Unit
|
|
||||||
|
|
||||||
1. Create a service file:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo nano /etc/systemd/system/onboarding-system.service
|
|
||||||
sudo nano /etc/systemd/system/nextcloud-sync.service
|
|
||||||
```
|
|
||||||
|
|
||||||
2. Add the following content to the file:
|
|
||||||
|
|
||||||
```ini
|
|
||||||
[Unit]
|
|
||||||
Description=Nextcloud Folder Sync Service
|
|
||||||
After=network.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
User=root
|
|
||||||
Environment="PATH=/root/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
|
||||||
WorkingDirectory=/path/to/your/project
|
|
||||||
ExecStart=/bin/bash -lc 'source /root/miniconda3/etc/profile.d/conda.sh && conda activate csv_pdf && /root/miniconda3/envs/csv_pdf/bin/python main.py'
|
|
||||||
Restart=always
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
```
|
|
||||||
and
|
|
||||||
|
|
||||||
```ini
|
|
||||||
[Unit]
|
|
||||||
Description=Nextcloud Folder Sync Service
|
|
||||||
After=network.target
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
User=root
|
|
||||||
Environment="PATH=/root/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
|
||||||
WorkingDirectory=/path/to/your/project
|
|
||||||
ExecStart=/bin/bash -lc 'source /root/miniconda3/etc/profile.d/conda.sh && conda activate csv_pdf && /root/miniconda3/envs/csv_pdf/bin/python sync_script.py'
|
|
||||||
Restart=always
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
```
|
|
||||||
|
|
||||||
3. Save the file and close it.
|
|
||||||
|
|
||||||
#### Step 2: Reload Systemd Daemon
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Step 3: Enable the Service
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl enable onboarding-system.service
|
|
||||||
sudo systemctl enable nextcloud-sync.service
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Step 4: Start the Service
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl start onboarding-system.service
|
|
||||||
sudo systemctl start nextcloud-sync.service
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Step 5: Check the Service Status
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl status onboarding-system.service
|
|
||||||
sudo systemctl status nextcloud-sync.service
|
|
||||||
```
|
|
||||||
|
|
||||||
### Logging
|
|
||||||
|
|
||||||
To view logs for the `nextcloud-sync` service, use the following command:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
journalctl -u nextcloud-sync.service
|
|
||||||
```
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This project is licensed under the MIT License. See the `LICENSE` file for details.
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
name: csv_pdf
|
|
||||||
channels:
|
|
||||||
- conda-forge
|
|
||||||
- defaults
|
|
||||||
dependencies:
|
|
||||||
- bzip2=1.0.8
|
|
||||||
- ca-certificates=2024.7.4
|
|
||||||
- libffi=3.4.2
|
|
||||||
- libsqlite=3.46.0
|
|
||||||
- libzlib=1.3.1
|
|
||||||
- openssl=3.3.1
|
|
||||||
- pip=24.2
|
|
||||||
- python=3.10.14
|
|
||||||
- setuptools=72.1.0
|
|
||||||
- tk=8.6.13
|
|
||||||
- ucrt=10.0.22621.0
|
|
||||||
- vc=14.3
|
|
||||||
- vc14_runtime=14.40.33810
|
|
||||||
- vs2015_runtime=14.40.33810
|
|
||||||
- wheel=0.44.0
|
|
||||||
- xz=5.2.6
|
|
||||||
- pip:
|
|
||||||
- backports-unittest-mock==1.5.1
|
|
||||||
- certifi==2024.8.30
|
|
||||||
- chardet==5.2.0
|
|
||||||
- charset-normalizer==3.3.2
|
|
||||||
- colorama==0.4.6
|
|
||||||
- docx2pdf==0.1.8
|
|
||||||
- idna==3.10
|
|
||||||
- imaplib2==3.6
|
|
||||||
- jinja2==3.1.4
|
|
||||||
- lxml==5.3.0
|
|
||||||
- markdown==3.6
|
|
||||||
- markupsafe==2.1.5
|
|
||||||
- mock==5.1.0
|
|
||||||
- numpy==2.0.1
|
|
||||||
- pandas==2.2.2
|
|
||||||
- pdfkit==1.0.0
|
|
||||||
- pdfrw==0.4
|
|
||||||
- pillow==10.4.0
|
|
||||||
- pypdf2==3.0.1
|
|
||||||
- python-dateutil==2.9.0.post0
|
|
||||||
- python-docx==1.1.2
|
|
||||||
- python-dotenv==1.0.1
|
|
||||||
- pytz==2024.1
|
|
||||||
- pywin32==306
|
|
||||||
- reportlab==4.2.2
|
|
||||||
- requests==2.32.3
|
|
||||||
- six==1.16.0
|
|
||||||
- tqdm==4.66.5
|
|
||||||
- typing-extensions==4.12.2
|
|
||||||
- tzdata==2024.1
|
|
||||||
- urllib3==2.2.3
|
|
||||||
|
|
||||||
Reference in New Issue
Block a user