From 872fff75b0944490acc0f85bab66e37f598cf82b Mon Sep 17 00:00:00 2001 From: "florin.hazi" Date: Wed, 19 Oct 2022 02:36:19 +0000 Subject: [PATCH 1/3] Update 'README.md' --- README.md | 163 +++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 113 insertions(+), 50 deletions(-) diff --git a/README.md b/README.md index b12ffac..2f31ce6 100644 --- a/README.md +++ b/README.md @@ -11,75 +11,138 @@ - Laravel Framework ^6.2 ## :rocket: Installation +**NOTE** + +The installation steps below were tested on an Ubuntu 20.04 LTS machine and should be adapted for each specific environment. + +--- + +### Update package repository +``` +sudo apt-get update -y +``` + +### Install Apache2 +``` +apt-get -y install \ + apache2 \ + apache2-doc \ + apache2-utils \ + libapache2-mod-fcgid +``` + +### Install PHP and the required extensions +``` +apt-get -y install software-properties-common && \ +add-apt-repository ppa:ondrej/php -y && \ +apt-get update -y && \ +apt-get -y install \ + php7.4 \ + php7.4-common \ + php7.4-fpm \ + php7.4-mbstring \ + php7.4-sqlite3 \ + php7.4-xml +``` + +### Configure Apache2 and PHP +``` +a2enmod \ + rewrite \ + actions \ + fcgid \ + alias \ + proxy_fcgi \ + remoteip && \ +sed -i "/^[[:blank:]]ErrorLog/i\ " /etc/apache2/sites-available/000-default.conf && \ +sed -i "/^[[:blank:]]ErrorLog/i\ SetHandler \"proxy:unix:\/var\/run\/php\/php7.4-fpm.sock|fcgi:\/\/localhost\"" /etc/apache2/sites-available/000-default.conf && \ +sed -i "/^[[[:blank:]]ErrorLog/i\ " /etc/apache2/sites-available/000-default.conf && \ +bash -c 'echo "RemoteIPHeader X-Forwarded-For" >> /etc/apache2/apache2.conf' && \ +sed -i "s/LogFormat \"%v:%p %h/LogFormat \"%v:%p %a/g" /etc/apache2/apache2.conf && \ +sed -i "s/LogFormat \"%h/LogFormat \"%a/g" /etc/apache2/apache2.conf && \ +chown -R www-data /var/www/html && \ +chmod -R 755 /var/www/html && \ +sed -i "s/AllowOverride None/AllowOverride All/g" /etc/apache2/apache2.conf && \ +systemctl restart apache2 +``` + +### Install Composer +`apt-get -y install composer` ### Ubuntu Packages -```bash +``` # LibreOffice -apt-add-repository ppa:libreoffice/ppa -apt-get update -apt-get install libreoffice - +apt-get update -y && \ +apt-add-repository -y ppa:libreoffice/ppa && \ +apt-get update -y && \ +apt-get install -y libreoffice +``` +``` # Python -apt-get update -apt-get install software-properties-common -add-apt-repository ppa:deadsnakes/ppa -apt-get install supervisor python3.8 python3.8-dev - +apt-get update -y && \ +apt-get install -y software-properties-common && \ +add-apt-repository -y ppa:deadsnakes/ppa && \ +apt-get install -y \ + build-essential \ + libpoppler-cpp-dev \ + pkg-config \ + supervisor \ + python3 \ + python3-dev +``` +``` # Redis -apt-get install redis-server - +apt-get install -y redis-server +``` +``` # PDF Convertor -apt-get install libpoppler-cpp-dev -apt-get install poppler-utils - +apt-get install -y \ + libpoppler-cpp-dev \ + poppler-utils +``` +``` # Tesseract OCR -add-apt-repository ppa:alex-p/tesseract-ocr-devel -apt-get update -apt-get install tesseract-ocr - +add-apt-repository -y ppa:alex-p/tesseract-ocr-devel && \ +apt-get update -y && \ +apt-get install -y tesseract-ocr +``` +``` # Unpaper -apt-get install unpaper - +apt-get install -y unpaper +``` +``` # DOCX to PDF Convertor -apt-get install unoconv - +apt-get install -y unoconv +``` +``` # Pandoc -apt-get install pandoc +apt-get install -y pandoc ``` ### Libraries Packages -```bash +``` # Pip -curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py -python get-pip.py -rm -rf get-pip.py -pip install --upgrade pip - +curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \ +python3 get-pip.py && \ +rm -rf get-pip.py && \ +pip3 install --upgrade pip +``` +``` # Pdftotext -pip install pdftotext - +pip3 install pdftotext +``` +``` # Supervisor -pip install supervisor -systemctl enable supervisor -mkdir /var/log/amqp +pip3 install supervisor && \ +systemctl enable supervisor && \ +mkdir /var/log/amqp && \ mkdir /var/log/queue - -# Deskew -cd DESKEW_INSTALLATION_DIRECTORY -cd Bin -./deskew INPUT OUTPUT - -# Dewarp -pip3 install opencv-python - -cd DEWARP_INSTALLATION_DIRECTORY -pip3 install -r requirements.txt ``` + ### Queues Supervisor config -Add a new Supervisor config file in the "/etc/supervisor/conf.d" path like in the example below: -Config file path: /etc/supervisor/conf.d/queue-worker-search-and-displace-ingest-production.conf +Config file path: **/etc/supervisor/conf.d/queue-worker-search-and-displace-ingest-production.conf** ```bash [program:queue-worker-search-and-displace-ingest-production] @@ -93,9 +156,9 @@ redirect_stderr=true stdout_logfile=/var/log/queue/queue-worker-search-and-displace-ingest-production.log ``` -The value for the 'command' key should reflect the app path (in the example above the app's path is "/var/www/html/searchanddisplace-ingest"). +The value for the **command** key should reflect the app path (in the example above the app's path is **/var/www/html/searchanddisplace-ingest**). -The 'stdout_logfile' value is the log file. All parent directories must already exist. +The **stdout_logfile** value is the log file. All parent directories must already exist. ### Install app ```bash From 7847c2dc0cffc5c4118f4d6a5b0d731fd2c5471d Mon Sep 17 00:00:00 2001 From: "florin.hazi" Date: Thu, 20 Oct 2022 10:54:23 +0000 Subject: [PATCH 2/3] Update 'README.md' --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 2f31ce6..215992a 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,8 @@ apt-get -y install \ php7.4-fpm \ php7.4-mbstring \ php7.4-sqlite3 \ - php7.4-xml + php7.4-xml \ + php7.4-zip ``` ### Configure Apache2 and PHP From 4ea9219b3f9599fa3dc8bf3aea7baac472375e1e Mon Sep 17 00:00:00 2001 From: "emilian.mitocariu" Date: Thu, 20 Oct 2022 14:56:59 +0000 Subject: [PATCH 3/3] Update 'README.md' --- README.md | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 215992a..5afc2cb 100644 --- a/README.md +++ b/README.md @@ -13,13 +13,15 @@ ## :rocket: Installation **NOTE** -The installation steps below were tested on an Ubuntu 20.04 LTS machine and should be adapted for each specific environment. +The installation steps below were tested on an Ubuntu 20.04 LTS machine, all commands assume sudo being used unless specified otherwise and should be adapted for each specific environment. + +Disk size for this service should be at least 10GB. --- ### Update package repository ``` -sudo apt-get update -y +apt-get update -y ``` ### Install Apache2 @@ -55,6 +57,7 @@ a2enmod \ alias \ proxy_fcgi \ remoteip && \ +sed -i "s/DocumentRoot \/var\/www\/html/DocumentRoot \/var\/www\/html\/searchanddisplace-ingest\/public/g" /etc/apache2/sites-available/000-default.conf && \ sed -i "/^[[:blank:]]ErrorLog/i\ " /etc/apache2/sites-available/000-default.conf && \ sed -i "/^[[:blank:]]ErrorLog/i\ SetHandler \"proxy:unix:\/var\/run\/php\/php7.4-fpm.sock|fcgi:\/\/localhost\"" /etc/apache2/sites-available/000-default.conf && \ sed -i "/^[[[:blank:]]ErrorLog/i\ " /etc/apache2/sites-available/000-default.conf && \ @@ -162,6 +165,17 @@ The value for the **command** key should reflect the app path (in the example ab The **stdout_logfile** value is the log file. All parent directories must already exist. ### Install app +- Download app + +``` +cd /var/www/html && \ +git clone https://git.law/newroco/searchanddisplace-ingest.git && \ +chown -R www-data:www-data searchanddisplace-ingest && \ +cd searchanddisplace-ingest +``` + +- Install and configure app + ```bash # Generate environment file cp .env.example .env @@ -172,12 +186,20 @@ composer install # Generate app key php artisan key:generate -# Change the value for the QUEUE_CONNECTION to redis, if it is not set already +# Change in .env the value for the QUEUE_CONNECTION to redis, if it is not set already # Deploy supervisor +supervisorctl reread +supervisorctl update supervisorctl start all ``` +- Check the queue worker is running + +``` +supervisorctl status +``` + ### Search and Displace Core Setup - Install the `Search and Displace Core` app, found here https://git.law/newroco/searchanddisplace-core - Get the URL of the `Search and Displace Core` app and add it to the `WEBHOOK_CORE_URL` variable in `.env`