Merge pull request '[UPD] refactoring requirements instalation' (#7) from dev into master

Reviewed-on: #7

README.md

@@ -1,45 +1,94 @@
 # zdc-tools
 
-Obsahuje skript pre vytvorenie lokálnej VM prostredníctvom [multipass.run](https://multipass.run) aplikácie.
+Obsahuje skripty pre vytvorenie a konfiguráciu lokálnej VM prostredníctvom [multipass.run](https://multipass.run) aplikácie.
 
 
 ## Predpoklady
 
 - Nainštalovaná aplikácia [multipass.run](https://multipass.run)
 - Nainštalovaný git
+- Vygenerovaný ssh kľúč (id_rsa)
 
 ## Inicializácia
 Stiahnutie skriptov a konfiguračných súborov do cieľového adresára `<WORKING_DIR>`.
 
-```
+```bash
 git clone https://gitea.egroup.sk/development/zdc-tools --depth 1 --branch master <WORKING_DIR>/zdc-tools
 ```
 
 ### Nastavenie oprávnení
 
-```
+```bash
 cd <WORKING_DIR>/zdc-tools
-chmod +x zdc-odoo-vm.sh
+chmod +x *.sh
 
 ```
 
 ## Vytvorenie a konfigurácia VM
 
 ### Popis skriptu
-Pri prvom spustení sa stiahne používaný image (Ubuntu 18.04 LTS).
+Pri prvom spustení `vm-create.sh` sa stiahne používaný image (Ubuntu 18.04 LTS).
 Následne skript prostredníctvom aplikácie `multipass` inicializuje virtuálny server prostredníctvom `./cloud-init/zdc-odoo-init.yaml` súboru.
 
-Prostrednícvom skriptu sú príkazmi `multipass exec` konfigurované jednotlivé časti prostredia:
+Prostredníctvom skriptu sú príkazmi `multipass exec` konfigurované jednotlivé časti prostredia:
 - python venv
 - stiahne zdc-odoo repozitár
 - vytvorí databázu podľa šablóny
 
 ### Parametre skriptu
-- `<ZDC_VM_NAME>` - názov virtuálneho servera
+- `<VM_NAME>` - názov virtuálneho servera
 
 
 ### Spustenie skriptu
-```
-./zdc-odoo-vm.sh <ZDC_VM_NAME>
+```bash
+./vm-create.sh <VM_NAME>
+
+```
+
+## Vytvorenie novej databázy
+
+### Popis skriptu
+Skript `vm-db-create.sh` umožňuje vytvoriť novú databázu zo šablóny.
+
+### Parametre skriptu
+- `<VM_NAME>` - názov existujúceho virtuálneho servera
+- `<DB_NAME>` - názov novej databázy
+
+### Spustenie skriptu
+```bash
+./vm-db-create.sh <VM_NAME> <DB_NAME>
+
+```
+
+## Aktualizácia kódu platformy z gitea repozitára
+
+### Popis skriptu
+Skript `vm-zdc-update.sh` umožňuje aktualizovať existujúce inštalácie **zdc-core-addons** a **zdc-oca-addons** z gitea repozitára.  
+
+### Parametre skriptu
+- `<VM_NAME>` - názov existujúceho virtuálneho servera
+
+### Spustenie skriptu
+```bash
+./vm-zdc-update.sh <VM_NAME>
+
+```
+
+## Aktualizácia development z gitea repozitára
+
+### Popis skriptu
+Pri novej inštalácii sa vývojový adresár inicializuje z **master** vetvy. Skript `vm-dev-update.sh` umožňuje aktualizovať vývojový adresár z **dev** vetvy gitea repozitára.  Pri vývoji si každý vývojár vytvorí vlastnú vetvu, v ktorej priebežne zapisuje svoje zmeny a následne  ju spojí do **dev** vetvy. Spojenie do **master** vetvy môže potvrdiť len administrátor.
+
+### Parametre skriptu
+- `<VM_NAME>` - názov existujúceho virtuálneho servera
+
+### Interné premenné
+Slúžia na vlastné prispôsobenie skriptu pre nastavenie aktuálne používaného repozitára pre vývoj.
+- `<DEV_REPO>` - názov používaného repozitára pre vývoj
+- `<DEV_BRANCH>` - názov používanej vetvy (branch)
+
+### Spustenie skriptu
+```bash
+./vm-dev-update.sh <VM_NAME>
 
 ```

cloud-init/zdc-odoo-init.yaml

@@ -10,6 +10,7 @@ packages:
 - curl
 - git
 - wget
+- unzip
 - python3-pip
 - build-essential
 - python3-dev
@@ -31,6 +32,7 @@ bootcmd:
 runcmd:
 # enable postgres
 - systemctl enable --now postgresql.service
+# create db user
 - su - postgres -c "createuser -s ubuntu"
 # setup directories
 - sudo -u ubuntu mkdir /home/ubuntu/config

config/oca_requirements.txt

@@ -0,0 +1,17 @@
+python-ldap   ## for ldap addons
+cerberus      ## for rest addons
+pyquerystring  ## for rest addons
+parse-accept-language  ## for rest addons
+apispec       ## for new rest addons
+marshmallow   ## for new rest addons
+marshmallow_objects  ## for new rest addons
+jsondiff      ## for new rest addons
+cachetools    ## for connector addons
+altair         ## for kpi dashboard addons
+bokeh          ## for kpi dashboard addons
+zxcvbn         ## for password security
+xlsxwriter     ## for xlsx export
+xlrd           ## for xlsx export
+sqlalchemy     ## for mysql connection
+lasso          ## for auth addon
+pyotp          ## for auth addons

config/odoo12.conf

@@ -3,4 +3,4 @@ db_host = False
 db_port = False
 db_user = ubuntu
 db_password = False
-addons_path = /home/ubuntu/odoo12/addons,/home/ubuntu/custom/zdc-oca-addons,/home/ubuntu/custom/dev-addons
+addons_path = /home/ubuntu/custom/zdc-core-addons,/home/ubuntu/custom/zdc-oca-addons,/home/ubuntu/custom/dev-addons

config/odoo_requirements.txt

@@ -0,0 +1,48 @@
+Babel==2.3.4
+chardet==3.0.4
+decorator==4.0.10
+docutils==0.12
+ebaysdk==2.1.5
+feedparser==5.2.1
+gevent==1.1.2 ; sys_platform != 'win32' and python_version < '3.7'
+gevent==1.5.0 ; python_version >= '3.7'
+greenlet==0.4.10 ; python_version < '3.7'
+greenlet==0.4.14 ; python_version >= '3.7'
+html2text==2016.9.19
+Jinja2==2.10.1
+libsass==0.12.3
+lxml==3.7.1 ; sys_platform != 'win32' and python_version < '3.7'
+lxml==4.2.3 ; sys_platform != 'win32' and python_version >= '3.7'
+lxml ; sys_platform == 'win32'
+Mako==1.0.4
+MarkupSafe==0.23
+mock==2.0.0
+num2words==0.5.6
+ofxparse==0.16
+passlib==1.6.5
+Pillow==4.0.0 ; python_version < '3.7'
+Pillow==6.1.0 ; python_version >= '3.7'
+psutil==4.3.1; sys_platform != 'win32'
+psutil==5.6.3; sys_platform == 'win32'
+psycopg2==2.7.3.1; sys_platform != 'win32'
+psycopg2==2.8.3; sys_platform == 'win32'
+pydot==1.2.3
+pyldap==2.4.28; sys_platform != 'win32'
+pyparsing==2.1.10
+PyPDF2==1.26.0
+pyserial==3.1.1
+python-dateutil==2.5.3
+pytz==2016.7
+pyusb==1.0.0
+qrcode==5.3
+reportlab==3.3.0
+requests==2.20.0
+suds-jurko==0.6
+vatnumber==1.2
+vobject==0.9.3
+Werkzeug==0.11.15 ; sys_platform != 'win32'
+Werkzeug==0.16.0  ; sys_platform == 'win32'
+XlsxWriter==0.9.3
+xlwt==1.3.*
+xlrd==1.0.0
+pypiwin32 ; sys_platform == 'win32'

config/venv-install.sh

@@ -76,4 +76,5 @@ if [ -n "${BASH:-}" -o -n "${ZSH_VERSION:-}" ] ; then
 fi
 
 pip3 install wheel
-pip3 install -r /home/ubuntu/odoo12/requirements.txt
+pip3 install -r /home/ubuntu/config/odoo_requirements.txt   ## for zdc-odoo
+pip3 install -r /home/ubuntu/config/oca_requirements.txt    ## for zdc-oca-addons

vm-create.sh

@@ -3,6 +3,9 @@
 NAME=$1
 
 PRIVATE_KEY=~/.ssh/id_rsa
+PUBLIC_KEY=~/.ssh/id_rsa.pub
+
+DB_NAME=zdcbasedb10
 
 echo "### Create Ubuntu 18.04 LTS VM"
 multipass launch bionic --name $NAME --cpus 2 --mem 2048M --disk 12G --cloud-init ./cloud-init/zdc-odoo-init.yaml
@@ -11,19 +14,23 @@ echo "### Install wkhtmltopdf"
 multipass exec $NAME -- sudo wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox_0.12.6-1.bionic_amd64.deb -O /tmp/wkhtmltox_0.12.6-1.bionic_amd64.deb
 multipass exec $NAME -- sudo apt install /tmp/wkhtmltox_0.12.6-1.bionic_amd64.deb -y
 
-echo "### Add ssh private key"
+echo "### Add ssh private and public key"
 multipass transfer $PRIVATE_KEY $NAME:/home/ubuntu/.ssh/id_rsa
+multipass transfer $PUBLIC_KEY $NAME:/home/ubuntu/.ssh/id_rsa.pub
 multipass transfer config/ssh_known_hosts $NAME:/home/ubuntu/.ssh/known_hosts
 multipass exec $NAME -- chmod 600 /home/ubuntu/.ssh/id_rsa
 
 echo "### Add  OCB Odoo, OCA addons"
 multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-odoo --depth 1 --branch master /home/ubuntu/odoo12
+multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-core-addons --depth 1 --branch master /home/ubuntu/custom/zdc-core-addons
 multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-oca-addons --depth 1 --branch master /home/ubuntu/custom/zdc-oca-addons
 multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-addons --depth 1 --branch master /home/ubuntu/custom/dev-addons
 
 echo "### Install Python Environment"
-multipass exec $NAME -- sudo -u ubuntu /usr/bin/python3 -m venv /home/ubuntu/venv-odoo12
 multipass transfer config/venv-install.sh $NAME:/home/ubuntu/config/venv-install.sh
+multipass transfer config/odoo_requirements.txt $NAME:/home/ubuntu/config/odoo_requirements.txt
+multipass transfer config/oca_requirements.txt $NAME:/home/ubuntu/config/oca_requirements.txt
+multipass exec $NAME -- sudo -u ubuntu /usr/bin/python3 -m venv /home/ubuntu/venv-odoo12
 multipass exec $NAME -- sudo -u ubuntu /bin/bash /home/ubuntu/config/venv-install.sh
 
 echo "### Setup OCB Odoo 12"
@@ -35,9 +42,13 @@ multipass exec $NAME -- sudo systemctl daemon-reload
 multipass exec $NAME -- sudo systemctl enable --now odoo12.service
 
 echo "### Create ZDC Template DB"
-multipass transfer db/zdc-template-db.sql $NAME:/home/ubuntu/db/zdc-template-db.sql
-multipass exec $NAME -- sudo -u postgres psql -c "CREATE DATABASE zdcbasedb10 WITH OWNER=ubuntu;"
-multipass exec $NAME -- sudo -u postgres psql -d zdcbasedb10 -f /home/ubuntu/db/zdc-template-db.sql
+multipass transfer db/zdc-db-dump-v1.sql.gz $NAME:/home/ubuntu/db/zdc-db-dump.sql.gz
+multipass transfer db/zdc-db-filestore-v1.zip $NAME:/home/ubuntu/db/zdc-db-filestore.zip
+multipass exec $NAME -- gzip -d /home/ubuntu/db/zdc-db-dump.sql.gz
+multipass exec $NAME -- sudo -u postgres psql -c "CREATE DATABASE $DB_NAME WITH OWNER=ubuntu;"
+multipass exec $NAME -- sudo -u postgres psql -d $DB_NAME -f /home/ubuntu/db/zdc-db-dump.sql
+multipass exec $NAME -- mkdir -p /home/ubuntu/.local/share/Odoo/filestore/$DB_NAME
+multipass exec $NAME -- unzip /home/ubuntu/db/zdc-db-filestore.zip -d /home/ubuntu/.local/share/Odoo/filestore/$DB_NAME
 
 echo "### Check VM"
 multipass list  | grep $NAME
@@ -45,3 +56,6 @@ URL_IP=`multipass info $NAME | grep IPv4 | sed -e 's/ //g' -e '/^$/d' | sed -e '
 
 echo "###"
 echo "### Odoo database manager on $NAME server: http://$URL_IP:8069/web/database/selector"
+echo "###"
+
+open http://$URL_IP:8069/web/

vm-db-create.sh

@@ -5,4 +5,6 @@ DB_NAME=$2
 
 echo "### Create Db from Template"
 multipass exec $NAME -- sudo -u postgres psql -c "CREATE DATABASE $DB_NAME WITH OWNER=ubuntu;"
-multipass exec $NAME -- sudo -u postgres psql -d $DB_NAME -f /home/ubuntu/db/zdc-template-db.sql
+multipass exec $NAME -- sudo -u postgres psql -d $DB_NAME -f /home/ubuntu/db/zdc-db-dump.sql
+multipass exec $NAME -- mkdir -p /home/ubuntu/.local/share/Odoo/filestore/$DB_NAME
+multipass exec $NAME -- unzip /home/ubuntu/db/zdc-db-filestore.zip -d /home/ubuntu/.local/share/Odoo/filestore/$DB_NAME

vm-dev-update.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+#
+# customise for your development fork
+
+NAME=$1
+DEV_REPO="development/zdc-addons"
+DEV_BRANCH="dev"
+
+echo "### Stop odoo service"
+multipass exec $NAME -- sudo systemctl stop odoo12.service
+
+echo "### Remove old dev repostitory"
+multipass exec $NAME -- sudo rm -r /home/ubuntu/custom/dev-addons
+
+echo "### Clone new dev repository"
+multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:$DEV_REPO --depth 1 --branch $DEV_BRANCH /home/ubuntu/custom/dev-addons
+
+echo "### Start odoo service"
+multipass exec $NAME -- sudo systemctl start odoo12.service

vm-zdc-update.sh

@@ -7,10 +7,12 @@ multipass exec $NAME -- sudo systemctl stop odoo12
 
 echo "### Remove old repostitories"
 multipass exec $NAME -- sudo rm -r /home/ubuntu/odoo12
+multipass exec $NAME -- sudo rm -r /home/ubuntu/custom/zdc-core-addons
 multipass exec $NAME -- sudo rm -r /home/ubuntu/custom/zdc-oca-addons
 
 echo "### Clone new repositories"
 multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-odoo --depth 1 --branch master /home/ubuntu/odoo12
+multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-core-addons --depth 1 --branch master /home/ubuntu/custom/zdc-core-addons
 multipass exec $NAME -- /usr/bin/git clone git@gitea.egroup.sk:development/zdc-oca-addons --depth 1 --branch master /home/ubuntu/custom/zdc-oca-addons
 
 echo "### Start odoo service"