Browse Source

Merge branch 'master' of ssh://ssh.git.law:2222/newroco/searchanddisplace-core

master
Radu Liviu Carjan 1 year ago
parent
commit
955aecce94
  1. 207
      README.md

207
README.md

@ -3,23 +3,112 @@
---
**NOTE**
The installation steps below were tested on an Ubuntu machine and should be adapted for each specific environment.
The installation steps below were tested on an Ubuntu 20.04 LTS machine, all commands assume sudo being used unless specified otherwise and should be adapted for each specific environment.
Disk size for this service should be at least 15GB.
---
## Install
### Update package repository
```
apt-get update -y
```
### Install Apache2
```
apt-get -y install \
apache2 \
apache2-doc \
apache2-utils \
libapache2-mod-fcgid
```
### Install PHP and the required extensions
```
apt-get -y install software-properties-common && \
add-apt-repository ppa:ondrej/php -y && \
apt-get update -y && \
apt-get -y install \
php7.4 \
php7.4-calendar \
php7.4-common \
php7.4-fileinfo \
php7.4-ftp \
php7.4-fpm \
php7.4-gettext \
php7.4-iconv \
php7.4-json \
php7.4-mbstring \
php7.4-opcache \
php7.4-pdo \
php7.4-phar \
php7.4-posix \
php7.4-readline \
php7.4-sockets \
php7.4-sqlite3 \
php7.4-tokenizer \
php7.4-xml
```
### Configure Apache2 and PHP
```
a2enmod \
rewrite \
actions \
fcgid \
alias \
proxy_fcgi \
remoteip && \
sed -i "s/DocumentRoot \/var\/www\/html/DocumentRoot \/var\/www\/html\/searchanddisplace-core\/public/g" /etc/apache2/sites-available/000-default.conf && \
sed -i "/^[[:blank:]]ErrorLog/i\ <FilesMatch \.php\$>" /etc/apache2/sites-available/000-default.conf && \
sed -i "/^[[:blank:]]ErrorLog/i\ SetHandler \"proxy:unix:\/var\/run\/php\/php7.4-fpm.sock|fcgi:\/\/localhost\"" /etc/apache2/sites-available/000-default.conf && \
sed -i "/^[[[:blank:]]ErrorLog/i\ </\FilesMatch>" /etc/apache2/sites-available/000-default.conf && \
bash -c 'echo "RemoteIPHeader X-Forwarded-For" >> /etc/apache2/apache2.conf' && \
sed -i "s/LogFormat \"%v:%p %h/LogFormat \"%v:%p %a/g" /etc/apache2/apache2.conf && \
sed -i "s/LogFormat \"%h/LogFormat \"%a/g" /etc/apache2/apache2.conf && \
chown -R www-data /var/www/html && \
chmod -R 755 /var/www/html && \
sed -i "s/AllowOverride None/AllowOverride All/g" /etc/apache2/apache2.conf && \
systemctl restart apache2
```
### Install Composer
```
apt-get -y install composer
```
### Install NodeJS 16 LTS , npm
```
curl -s https://deb.nodesource.com/setup_16.x | sudo bash
```
```
apt-get -y install \
nodejs \
yarn
```
### Install and Configure the app
- Download the app
```
cd /var/www/html && \
git clone https://git.law/newroco/searchanddisplace-core.git && \
chown -R www-data:www-data searchanddisplace-core && \
cd searchanddisplace-core
```
- Create the `.env` file by copying the contents from the `.env.example` file.
`cp .env.example .env`
- Install the 'sqlite' driver for your PHP version if it is not already installed.
- For the 'QUEUE_CONNECTION' variable in `.env` you can use either `sync` or `redis` (recommended). If you choose to use `redis`
then you need to make sure that it is installed on your machine.
`apt update`
`apt install redis-server`
```
apt-get -y install redis-server
```
- Install the `Search and Displace Ingest` app, found here https://git.law/newroco/searchanddisplace-ingest
- Get the URL of the `Search and Displace Ingest` app and add it to the `SD_INGEST_URL` variable in `.env`
@ -28,44 +117,33 @@ the `Search and Displace Ingest` app `.env` file
- Add in `.env` the `SD_DUCKLING_URL` value which by default is `http://0.0.0.0:8000/parse`. You can find
details about installing Facebook Duckling in a section below.
- Install composer
- Install composer dependencies
`composer install`
- Install NodeJS and npm
- Install npm dependencies
`npm install`
```
rm -rf node_modules && \
npm install
```
- Compile frontend assets
`npm run prod`
`npm run production`
- Generate the app key by running the following command:
`php artisan key:generate`
- Migrate DB tables
`php artisan migrate`
- Start Supervisor (after adding the Supervisor configs detailed below)
`supervisorctl start all`
- (Optional) Restart Supervisor after a config file update
`supervisorctl reread`
`supervisorctl update`
`supervisorctl restart <name>`
```
touch ./database/database.sqlite
chown www-data:www-data ./database/database.sqlite
php artisan migrate
```
### Queues Supervisor config
Add a new Supervisor config file in the "/etc/supervisor/conf.d" path like in the example below:
- Install supervisor
`apt-get install supervisor -y`
Config file path: /etc/supervisor/conf.d/queue-worker-search-and-displace-core-production.conf
- Config file path: **/etc/supervisor/conf.d/queue-worker-search-and-displace-core-production.conf**
```bash
[program:queue-worker-search-and-displace-core-production]
@ -79,24 +157,38 @@ redirect_stderr=true
stdout_logfile=/var/log/queue/queue-worker-search-and-displace-core-production.log
```
The value for the 'command' key should reflect the app path (in the example above the app's path is "/var/www/html/searchanddisplace-core").
The value for the **command** key should reflect the app path (in the example above the app's path is **/var/www/html/searchanddisplace-core**).
The **stdout_logfile** value is the log file. All parent directories must already exist.
` mkdir /var/log/queue`
- Start Supervisor (after adding the Supervisor configs detailed below)
`supervisorctl start all`
The 'stdout_logfile' value is the log file. All parent directories must already exist.
- (Optional) Restart Supervisor after a config file update
```
supervisorctl reread
supervisorctl update
supervisorctl restart <name>
```
### Facebook Duckling
- `$ apt-get install libpcre3-dev`
- Go to the directory in which you want to deploy the app (e.g. /var/www/html)
- `$ git clone https://github.com/facebook/duckling.git`
- `$ cd duckling`
- `$ curl -sSL https://get.haskellstack.org/ | sh`
- `$ stack build`
- `$ stack exec duckling-example-exe`
- `$ stack test`
```
apt-get -y install \
libpcre3 \
libpcre3-dev \
pkg-config && \
cd /var/www/html && \
git clone https://github.com/facebook/duckling.git fb-duckling && \
cd fb-duckling && \
curl -sSL https://get.haskellstack.org/ | sh && \
stack build && \
stack exec duckling-example-exe && \
stack test
```
### Facebook Duckling Supervisor config
Add a new Supervisor config file in the "/etc/supervisor/conf.d" path like in the example below:
Config file path: /etc/supervisor/conf.d/duckling-worker-search-and-displace-core-production.conf
Config file path: **/etc/supervisor/conf.d/duckling-worker-search-and-displace-core-production.conf**
```bash
[program:duckling-worker-search-and-displacecore-production]
@ -110,17 +202,32 @@ numprocs=1
redirect_stderr=true
stdout_logfile=/var/log/queue/duckling-worker-search-and-displace-core-production.log
```
The value for the 'directory' key should reflect the Facebook Duckling app path (in the example above the path is "/var/www/html/fb-duckling").
The value for the **directory** key should reflect the Facebook Duckling app path (in the example above the path is **/var/www/html/fb-duckling**).
The **stdout_logfile** value is the log file. All parent directories must already exist.
The 'stdout_logfile' value is the log file. All parent directories must already exist.
### Start the queue worker and Facebook Duckling with Supervisor
```
supervisorctl reread
supervisorctl update
supervisorctl start all
```
- Check they are running
```
supervisorctl status
```
### Converting documents
```bash
```
# LibreOffice
apt-get install software-properties-common
apt-add-repository ppa:libreoffice/ppa
apt-get update
apt-get install libreoffice
apt-get install -y software-properties-common && \
apt-add-repository ppa:libreoffice/ppa && \
apt-get update && \
apt-get install -y libreoffice
```
# Searchers

Loading…
Cancel
Save