Diff for "NodusUpgradeNov2017"

Differences between revisions 8 and 28 (spanning 20 versions)

Contents

OLD nodus etc / export
iptables
Apache
elogd
SVN

Described by KA on Nov 21st, 2017.

OLD nodus etc / export

Backup of the old nodus configurations are found in /cvs/cds/caltech/nodus_backup .

Some useful locations:

Old apache configurations

cd /cvs/cds/caltech/nodus_backup/etc/apache2

iptables

Firewall setting of nodus is now configured via shorewall. Shorewall produces the commands for iptables. Note: shorewall is a (sort of a) wapper for iptables, and is not a daemon.

On May 31, 2018, Jonathan Hanks and Koji Arai setup shorewall to set the iptables automatically when the machine is booted (although this was not tested).

The following command is to manually turn on the firewall settings

sudo systemctl start shorewall

Apache

Global settings

- To work with the server daemon, use systemctl command.

sudo systemctl (start|restart|stop|status) httpd

- All apache setting can be found in /etc/httpd .

- The main config file is /etc/httpd/conf/ . This takes care the global setting of the web service.

- PHP is required for dokuwikis. So it was installed. We need more careful investigation of the php configuration later.

sudo yum -y install php

SSL

- /etc/httpd/conf.d/ has sub config files. Only ssl.conf was reviewed. This config has the following line.

SSLPassPhraseDialog exec:/etc/httpd/passphrase.sh

And the corresponding file can be found as /etc/httpd/passphrase.sh .

- SSL Certificates were copied from the old backup. They are located in /etc/httpd/ssl. So each https sites should have the following directives in each VirtualHost.

SSLCertificateFile /etc/httpd/ssl/nodus.ligo.caltech.edu.crt
SSLCertificateKeyFile /etc/httpd/ssl/nodus.ligo.caltech.edu.key

Site files

- The actual web site settings are found in /etc/httpd/site-enabled . This folder should have symbolic links to the corresponding files in site-available. The important files are elog.conf and nodus30889_secure.conf for now. SVN setting should be added later.

- The site on the port 30889 is configured by nodus30889_secure.conf. This is a normal web server looking at the files in /export/home, not in /users/public_html!. One who wants to add a folder to the web is to make a link to the folder in /export/home and add a corresponding Directory entry in nodus30889_secure.conf.

- elog.conf takes care of redirecting and proxy of 8081 (https) to 8080. The line Header add Strict-Transport-Security "max-age=0" is important so that the client browsers does not remember 8080 as https.

- svn.conf takes care of svn. In this file, the location of the authentication files are speficied.

    AuthUserFile /export/svn/.svn-auth-file
    AuthzSVNAccessFile /export/svn/svn.authz

These files are copied from the old svn backup.

elogd

- The new executable was installed (via RPM by Rana) at /usr/local/sbin. The nominal setting files are in /export/elog/elog-common.

- The current (best) elog staring script is /export/elog/startELOGD.sh. This does not kill running process. Some sophisticated mechanism (as before) will be useful. Note that the 8080 port has to be opened manually for this to work - see iptables section above.

- Riddle: elogd does not want to use the themes in /export/elog/elog-common no matter how this directory is specified for the resource dir. The only workable setup right now is to speficy the resource directory as

Resource dir = /usr/local/elog

in the main config file /export/elog/elog-common/elogd.cfg.

SVN

Installation

SVN installation

sudo yum install subversion mod_dav_svn

Repository migration

To create a repository from an existing repository: We can't just copy the files. We need to use a dump file created by svnadmin. This can be done for an existing svn repository in the new system.

Dumping

cd /home/export/svn
sudo svnadmin dump ./ > ../svndump

Create a new repository

cd /home/export
sudo mv svn svn_old
sudo svnadmin create svn

Loading

cd /home/export
sudo svnadmin load ./svn < svndump

Apache setting

- See above (Site files).

svn server daemon

We don't need to run "svnserve" to use WEBDAV interface.

- To launch svn server run the following command

sudo svnserve -d

This allows us to access to the usual svn command from remote and https://nodus.ligo.caltech.edu:30889/svn/.

websvn

- Download the latest distribution

cd /cvs/cds/caltech/users/public_html
wget http://websvn.tigris.org/files/documents/1380/49056/websvn-2.3.3.tar.gz
tar zxvf websvn-2.3.3.tar.gz

- Expose the downloaded file as "websvn"

cd /export/home
ln -s /cvs/cds/caltech/users/public_html/websvn-2.3.3 ./websvn

- Edit configuration to specify the location of the repository

cd websvn/include
mv distconfig.php config.php
emacs -nw config.php

Add the following line right next to "// Local repositories (without and with optional group)"

$config->addRepository('40m SVN', 'file:///export/svn/', NULL, 'svn40m', '0p2iCs');

-  ⇤ ← Revision 8 as of 2017-11-22 01:16:21 → 
  Size: 6345
  Editor: KojiaraiATligoDOTorg
  Comment:
+   ← Revision 28 as of 2018-06-01 03:40:13 → ⇥
  Size: 5553
  Editor: KojiaraiATligoDOTorg
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-<<TableOfContents(2)>>
+<<TableOfContents(3)>>
 Line 18:
-We want nodus to make several ports available from Martian and internet. Otherwise, the access to these ports are just rejected. To enable the ports, allow this, we need to run the following commands. 8080: non-secure elog, 8081: secure-elog, 30889: web service
+Firewall setting of nodus is now configured via '''shorewall'''. Shorewall produces the commands for iptables. Note: shorewall is a (sort of a) wapper for iptables, and is not a daemon.
 Line 20:
+On May 31, 2018, Jonathan Hanks and Koji Arai setup shorewall to set the iptables automatically when the machine is booted (although this was not tested).

The following command is to manually turn on the firewall settings
-Line 21:
+Line 24:
-sudo iptables -I INPUT 5 -i enp1s0f0 -p tcp --dport 8080 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo iptables -I INPUT 6 -i enp1s0f1 -p tcp --dport 8080 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo iptables -I INPUT 7 -i enp1s0f0 -p tcp --dport 30889 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo iptables -I INPUT 8 -i enp1s0f1 -p tcp --dport 30889 -m state --state NEW,ESTABLISHED -j ACCEPT
sudo iptables -I INPUT 9 -i enp1s0f0 -p tcp --dport 8081 -m state --state NEW,ESTABLISHED -j ACCEPT 
sudo iptables -I INPUT 10 -i enp1s0f1 -p tcp --dport 8081 -m state --state NEW,ESTABLISHED -j ACCEPT 
}}}

Here we need to specify two ethernet interfaces (enp1s0f0 and enp1s0f1). They could be checked by {{{ifconfig}}}

{{{
ifconfig -a
}}}

To check the current IPTable setup, the following command is useful
{{{
sudo iptables -vnL INPUT --line
}}}
The output of this should look like this
{{{
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    1005K 1464M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
2      889 49872 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0
3    5772K 3236M INPUT_direct  all  --  *      *       0.0.0.0/0            0.0.0.0/0
4    5772K 3236M INPUT_ZONES_SOURCE  all  --  *      *       0.0.0.0/0            0.0.0.0/0
5        0     0 ACCEPT     tcp  --  enp1s0f0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 state NEW,ESTABLISHED
6     1565 97196 ACCEPT     tcp  --  enp1s0f1 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 state NEW,ESTABLISHED
7        0     0 ACCEPT     tcp  --  enp1s0f0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:30889 state NEW,ESTABLISHED
8      412 23688 ACCEPT     tcp  --  enp1s0f1 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:30889 state NEW,ESTABLISHED
9        0     0 ACCEPT     tcp  --  enp1s0f0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8081 state NEW,ESTABLISHED
10    2884  175K ACCEPT     tcp  --  enp1s0f1 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8081 state NEW,ESTABLISHED
11   5767K 3236M INPUT_ZONES  all  --  *      *       0.0.0.0/0            0.0.0.0/0
12     370 15354 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate INVALID
13   5586K 3192M REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-host-prohibited
}}}

To delete an entry (e.g. #3 in INPUT section)
{{{
sudo iptables -D INPUT 3
+sudo systemctl start shorewall
-Line 64:
+Line 28:
-- This command makes the change permanent.'''This command has already been run'''.
{{{
iptables-save > /etc/sysconfig/iptables
}}}
-Line 70:
+Line 34:
+=== Global settings ===

- To work with the server daemon, use systemctl command.

{{{
sudo systemctl (start|restart|stop|status) httpd
}}}
-Line 74:
+Line 46:
+- PHP is required for dokuwikis. So it was installed. We need more careful investigation of the php configuration later.

{{{
sudo yum -y install php
}}}

=== SSL ===
-Line 83:
+Line 63:
-- The actual web site settings are found in {{{/etc/httpd/site-enabled}}} .  '''This folder should have symbolic links to the corresponding files in''' {{{site-available}}}. The important files are {{{elog.conf}}} and {{{nodus30889_secure.conf}}} for now. SVN setting should be added later.
-Line 88:
+Line 66:
-        SSLCertificateFile /etc/httpd/ssl/nodus.ligo.caltech.edu.crt
        SSLCertificateKeyFile /etc/httpd/ssl/nodus.ligo.caltech.edu.key
+SSLCertificateFile /etc/httpd/ssl/nodus.ligo.caltech.edu.crt
SSLCertificateKeyFile /etc/httpd/ssl/nodus.ligo.caltech.edu.key
-Line 92:
+Line 70:
+=== Site files ===
-Line 93:
+Line 72:
+- The actual web site settings are found in {{{/etc/httpd/site-enabled}}} .  '''This folder should have symbolic links to the corresponding files in''' {{{site-available}}}. The important files are {{{elog.conf}}} and {{{nodus30889_secure.conf}}} for now. SVN setting should be added later.
-Line 94:
+Line 74:
+- The site on the port 30889 is configured by {{{nodus30889_secure.conf}}}. This is a normal web server looking at the files in {{{/export/home}}}, not in {{{/users/public_html}}}!. One who wants to add a folder to the web is to make a link to the folder in {{{/export/home}}} and add a corresponding {{{Directory}}} entry in {{{nodus30889_secure.conf}}}.
-Line 95:
+Line 76:
+- {{{elog.conf}}} takes care of redirecting and proxy of 8081 (https) to 8080. The line
{{{Header add Strict-Transport-Security "max-age=0"}}}
is important so that the client browsers '''does not''' remember 8080 as https.
-Line 96:
+Line 80:
-- Now I configured non-SSL version of 30889 server. This was done by /etc/httpd/sites-available/nodus30889_nosecure.conf and the symbolic link of it in /etc/httpd/sites-enabled.
+- {{{svn.conf}}} takes care of svn. In this file, the location of the authentication files are speficied.
-Line 98:
+Line 82:
-- I realized that we need php for dokuwikis. It was installed. Maybe we need more careful configuration of php later.
> sudo yum -y install php
+{{{
    AuthUserFile /export/svn/.svn-auth-file
    AuthzSVNAccessFile /export/svn/svn.authz
}}}
-Line 101:
+Line 87:
-- To work with the server daemon, use systemctl command. Currently the server is not running.
> sudo systemctl (start|restart|stop|status) httpd
+These files are copied from the old svn backup.
-Line 106:
+Line 91:
-- The new executable seemed installed at /usr/local/sbin. The setting files are in /export/elog.
+- The new executable was installed (via RPM by Rana) at /usr/local/sbin. The nominal setting files are in /export/elog/elog-common.
-Line 108:
+Line 93:
-- The current (best) elog staring script is /export/elog/startELOGD.sh.
+- The current (best) elog staring script is /export/elog/startELOGD.sh. '''This does not kill running process. Some sophisticated mechanism (as before) will be useful. ''' ''Note that the 8080 port has to be opened manually for this to work - see iptables section above.''
-Line 110:
+Line 95:
-- By disabling 8081 secure elog thing, the elog is running at the port 8080. However, the elogd does not want to use the themes in /export/elog/elog-common no matter how this directory is specified for the resource dir. The only workable setup right now is to speficy the resource directory as
+- '''Riddle:''' {{{elogd}}} does not want to use the themes in /export/elog/elog-common no matter how this directory is specified for the resource dir. The only workable setup right now is to speficy the resource directory as

{{{
-Line 112:
+Line 99:
-in
/export/elog/elog-common/elogd.cfg
+}}}
-Line 115:
+Line 101:
-- This is not the perfect solution but this allows us to use the elog. There is no true secure password for the elog, this is OK for today, I guess? We need more investigation on the theme and the SSL version of the elog (i.e. port 8081).


ELOG CFG
URL = https://nodus.ligo.caltech.edu:8081/
+in the main config file {{{/export/elog/elog-common/elogd.cfg}}}.
-Line 123:
+Line 105:
+=== Installation ===
-Line 125:
+Line 109:
-sudo yum install subversion
+sudo yum install subversion mod_dav_svn
-Line 127:
+Line 111:
+=== Repository migration ===
-Line 143:
+Line 129:
-Line 149:
+Line 134:
+=== Apache setting ===

- See above ([[https://wiki-40m.ligo.caltech.edu/NodusUpgradeNov2017#Site_files |Site files]]). 

=== svn server daemon ===

We don't need to run "svnserve" to use WEBDAV interface.

- --('''To launch svn server run the following command ''')--

--(sudo svnserve -d)--

--(This allows us to access to the usual svn command from remote and [[https://nodus.ligo.caltech.edu:30889/svn/]]. )--

=== websvn ===

- Download the latest distribution

{{{
cd /cvs/cds/caltech/users/public_html
wget http://websvn.tigris.org/files/documents/1380/49056/websvn-2.3.3.tar.gz
tar zxvf websvn-2.3.3.tar.gz
}}}

- Expose the downloaded file as "websvn"

{{{
cd /export/home
ln -s /cvs/cds/caltech/users/public_html/websvn-2.3.3 ./websvn
}}}

- Edit configuration to specify the location of the repository

{{{
cd websvn/include
mv distconfig.php config.php
emacs -nw config.php
}}}

Add the following line right next to "// Local repositories (without and with optional group)"

{{{
$config->addRepository('40m SVN', 'file:///export/svn/', NULL, 'svn40m', '0p2iCs');
}}}