@ -11,75 +11,142 @@
- Laravel Framework ^6.2
## :rocket: Installation
**NOTE**
The installation steps below were tested on an Ubuntu 20.04 LTS machine, all commands assume sudo being used unless specified otherwise and should be adapted for each specific environment.
Disk size for this service should be at least 10GB.
---
### Update package repository
```
apt-get update -y
```
### Install Apache2
```
apt-get -y install \
apache2 \
apache2-doc \
apache2-utils \
libapache2-mod-fcgid
```
### Install PHP and the required extensions
```
apt-get -y install software-properties-common & & \
add-apt-repository ppa:ondrej/php -y & & \
apt-get update -y & & \
apt-get -y install \
php7.4 \
php7.4-common \
php7.4-fpm \
php7.4-mbstring \
php7.4-sqlite3 \
php7.4-xml \
php7.4-zip
```
### Configure Apache2 and PHP
```
a2enmod \
rewrite \
actions \
fcgid \
alias \
proxy_fcgi \
remoteip & & \
sed -i "s/DocumentRoot \/var\/www\/html/DocumentRoot \/var\/www\/html\/searchanddisplace-ingest\/public/g" /etc/apache2/sites-available/000-default.conf && \
sed -i "/^[[:blank:]]ErrorLog/i\ < FilesMatch \.php \$> " /etc/apache2/sites-available/000-default.conf && \
sed -i "/^[[:blank:]]ErrorLog/i\ SetHandler \"proxy:unix:\/var\/run\/php\/php7.4-fpm.sock|fcgi:\/\/localhost\"" /etc/apache2/sites-available/000-default.conf && \
sed -i "/^[[[:blank:]]ErrorLog/i\ </ \FilesMatch > " /etc/apache2/sites-available/000-default.conf && \
bash -c 'echo "RemoteIPHeader X-Forwarded-For" >> /etc/apache2/apache2.conf' & & \
sed -i "s/LogFormat \"%v:%p %h/LogFormat \"%v:%p %a/g" /etc/apache2/apache2.conf && \
sed -i "s/LogFormat \"%h/LogFormat \"%a/g" /etc/apache2/apache2.conf && \
chown -R www-data /var/www/html & & \
chmod -R 755 /var/www/html & & \
sed -i "s/AllowOverride None/AllowOverride All/g" /etc/apache2/apache2.conf & & \
systemctl restart apache2
```
### Install Composer
`apt-get -y install composer`
### Ubuntu Packages
```bash
```
# LibreOffice
apt-add-repository ppa:libreoffice/ppa
apt-get update
apt-get install libreoffice
apt-get update -y & & \
apt-add-repository -y ppa:libreoffice/ppa & & \
apt-get update -y & & \
apt-get install -y libreoffice
```
```
# Python
apt-get update
apt-get install software-properties-common
add-apt-repository ppa:deadsnakes/ppa
apt-get install supervisor python3.8 python3.8-dev
apt-get update -y & & \
apt-get install -y software-properties-common & & \
add-apt-repository -y ppa:deadsnakes/ppa & & \
apt-get install -y \
build-essential \
libpoppler-cpp-dev \
pkg-config \
supervisor \
python3 \
python3-dev
```
```
# Redis
apt-get install redis-server
apt-get install -y redis-server
```
```
# PDF Convertor
apt-get install libpoppler-cpp-dev
apt-get install poppler-utils
apt-get install -y \
libpoppler-cpp-dev \
poppler-utils
```
```
# Tesseract OCR
add-apt-repository ppa:alex-p/tesseract-ocr-devel
apt-get update
apt-get install tesseract-ocr
add-apt-repository -y ppa:alex-p/tesseract-ocr-devel & & \
apt-get update -y & & \
apt-get install -y tesseract-ocr
```
```
# Unpaper
apt-get install unpaper
apt-get install -y unpaper
```
```
# DOCX to PDF Convertor
apt-get install unoconv
apt-get install -y unoconv
```
```
# Pandoc
apt-get install pandoc
apt-get install -y pandoc
```
### Libraries Packages
```bash
```
# Pip
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
rm -rf get-pip.py
pip install --upgrade pip
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py & & \
python3 get-pip.py & & \
rm -rf get-pip.py & & \
pip3 install --upgrade pip
```
```
# Pdftotext
pip install pdftotext
pip3 install pdftotext
```
```
# Supervisor
pip install supervisor
systemctl enable supervisor
mkdir /var/log/amqp
pip3 install supervisor & & \
systemctl enable supervisor & & \
mkdir /var/log/amqp & & \
mkdir /var/log/queue
# Deskew
cd DESKEW_INSTALLATION_DIRECTORY
cd Bin
./deskew INPUT OUTPUT
# Dewarp
pip3 install opencv-python
cd DEWARP_INSTALLATION_DIRECTORY
pip3 install -r requirements.txt
```
### Queues Supervisor config
Add a new Supervisor config file in the "/etc/supervisor/conf.d" path like in the example below:
Config file path: /etc/supervisor/conf.d/queue-worker-search-and-displace-ingest-production.conf
Config file path: **/etc/supervisor/conf.d/queue-worker-search-and-displace-ingest-production.conf**
```bash
[program:queue-worker-search-and-displace-ingest-production]
@ -93,11 +160,22 @@ redirect_stderr=true
stdout_logfile=/var/log/queue/queue-worker-search-and-displace-ingest-production.log
```
The value for the 'command' key should reflect the app path (in the example above the app's path is "/var/www/html/searchanddisplace-ingest" ).
The value for the **command** key should reflect the app path (in the example above the app's path is **/var/www/html/searchanddisplace-ingest**
The 'stdout_logfile' value is the log file. All parent directories must already exist.
The **stdout_logfile**
### Install app
- Download app
```
cd /var/www/html & & \
git clone https://git.law/newroco/searchanddisplace-ingest.git & & \
chown -R www-data:www-data searchanddisplace-ingest & & \
cd searchanddisplace-ingest
```
- Install and configure app
```bash
# Generate environment file
cp .env.example .env
@ -108,12 +186,20 @@ composer install
# Generate app key
php artisan key:generate
# Change the value for the QUEUE_CONNECTION to redis, if it is not set already
# Change in .env the value for the QUEUE_CONNECTION to redis, if it is not set already
# Deploy supervisor
supervisorctl reread
supervisorctl update
supervisorctl start all
```
- Check the queue worker is running
```
supervisorctl status
```
### Search and Displace Core Setup
- Install the `Search and Displace Core` app, found here https://git.law/newroco/searchanddisplace-core
- Get the URL of the `Search and Displace Core` app and add it to the `WEBHOOK_CORE_URL` variable in `.env`
Radu Liviu Carjan
2 years ago