AutoMISP-CentOS/RHEL Edition

The MISP Project is probably one of the most popular Threat Intelligence platforms in recent memory. Its highly customizable, easy to use, and both easy to pull data from and give data to in a variety of different formats.

The Installation process takes a little bit of getting used to however. Thats where I come in. With this handy shell script.

What is the purpose of this script?

To leave you with a mostly functional MISP Installation on CentOS or Redhat Linux. This script has been successfully tested on both CentOS7 and RHEL7. At this time however, I cannot offer support for RHEL8. There is just too much in flux right now, and way too many major differences from RHEL/CentOS7. Package repos like EPEL are still in the process of being updated, and at least some functionality simply does not work on RHEL8 per the official MISP Install documentation for RHEL8. Be patient. It'll come with time.

What, Specifically, does this script do?

A lot. 99% of it comes directly from the RHEL7/CentOS 7 Installation guide itself, with little things I had to add in order for the instructions to actually work. Rather than list everything out, let me link you to the Install guide for CentOS/RHEL 7: https://misp.github.io/MISP/INSTALL.rhel7/ . Just about the only thing this script does NOT do that the installation guide covers is the section MISP Dashboard on CentOS

Instructions for use:

1. Download this repo via git, or download the .zip file that github will provide you. The repo comes with a file automisp.conf, that you'll want to review and/or modify however you see fit. The various configuration options are well-documented and commented, and the defaults are pretty sensible. If you're okay with your MISP instance hostname being 'misp.local' and the 32-character alphanumeric passwords generated by default, you can skip to step 2.
   • RHEL7 users: Before you begin, there are a couple of prereqs you need to have in place. first and foremost, verify your RHEL system has an active support subscription. you'll need to register your system via subscription-manager register --auto-attach to ensure your system has access to the correct RPM repos.
   • Also for RHEL7 users: You need to have or request access to the Redhat Sofware Collection (RHSCL) repository before you can successfully run this script. If you have no idea what I'm talking about, you need to ensure that your redhat support subscription either includes access to RHSCL, or make a request to gain access to the RHSCL repos. Check out this link for more info: https://access.redhat.com/solutions/472793 . Once you have that access sorted, AutoMISP can handle subscribing your system to the rhscl repo as a part of AutoMISP
   • Yet another note for RHEL7 users: do you just want to play around on RHEL7? But don't have money? check out the redhat developer portal, and create an account there https://developers.redhat.com/ . This subscription is NOT inteded for enterprise use, but DOES include access to the RHSCL repo. Please don't abuse developer licenses.
2. Once you have reviewed the automisp.conf file, and are satisfied with your configuration settings, simply start the autoMISP-RHEL-CentOS.sh shell script. Be aware that due to the number of configuration changes we will be making, that you will need to run the script as the root user or via the sudo command.
   • you should be able to run the script via bash autoMISP-RHEL-CentOS.sh if you are the root user, or via sudo bash autoMISP-RHEL-CentOS.sh as a non-root user (provided you have sudo privileges)
   • If you choose to operate with the default settings, your MISP server will have its hostname changed to misp.local. you can change this by editing the FQDN variable
   • The script will create a system user account, that is named misp by default, and defined by the MISP_USER variable. Its password is a randomly generated 32-character alphanumeric password that is generated when the script is executed. The password for the user account is saved to the file /root/misp_creds and is only accessible by the root user (chmod 600 permissions)
   • The script also sets the password for the root mysql user, as well as an misp database user that is used to administer the misp database, that the MISP web application relies heavily on. Just like with the misp system user, the passwords are both randomly generated 32-character alphanumeric strings that are saved to the file /root/misp_creds
   • This is merely personal suggestion, but I highly recommend copying all of the credentials out of the /root/misp_creds file, and saving them to something like a password manager for safe keeping.
3. The script will take a long time to run. Its no exaggeration to say that this will probably take a full hour to run to completion There are several configuration prerequisites that require significant time, processing power, or bandwidth to run such as:
   • Installing a metric ton of pre-requisite packages from the EPEL/SCL repos. There are well over 800+MB of packages that this script downloads. Expect this to take about 15 minutes or so on a decent connection.
   • Installing LIEF/pyLIEF and compiling it from source. In my experiences, this has taken over 20 minutes to compile on my VMs
   • Generating a dhparam.pem file as a part of good SSL housekeeping. This took my VM around 20 minutes or so to do, even with haveged running to help with entropy/RNG.
     • Yes, the script installs and configures a self-signed SSL cert.
     • Yes, I realize this is a contradiction.
     • At some point, I'm considering supporting getting an SSL cert via letsencrypt and acme.sh
4. If you would like to see the output from various commands while the script is running them, you can open a second terminal window and run the command tail -f /var/log/misp_install.log
   • Note that if you run into problems with the automisp script, that this log file will help you immensely with figuring out which command(s) failed and why.
5. Upon completion, you should have a (mostly) working MISP installation. Point your favorite web browser to https://[your ip adress here] and you should be greeted with the MISP login prompt. Default credentials for MISP are username: [email protected] password: admin upon first login, you will immediately be prompted to change this password.
   • After confirming successful login and execution of the script, I recommend documenting the password of the misp web admin, the misp user, as well as the credentials for the mysql root and misp users.
   • After you have documented the username and credentials, I recommend either archiving or deleting the /root/misp_creds file along with the /var/log/misp_install.log file, because both the creds file as well as the installer log reveal sensitive configuration information AND misp credentials. Keep it secret, keep it safe.
6. If you need to restart the misp workers, the systemd service misp-workers.service supports the stop, start, and restart actions.
7. Likewise if you need to restart/reconfigure the misp-modules, you can use the systemd service misp-modules.service. It also supports the stop,start, and restart actions.

Known Issues

As of this writing, Administration > Server Settings & Maintenance > Workers doesn't work correctly. At this point, I'm not sure if its a problem that MISP just can't find the PIDs, or if the workers are actually dead and don't come back when you restart them via the webUI.

If you attempt to restart a worker via the webUI, MISP marks the worker as red and claims that its dead. I'm not actually sure that its dead, because none of the logs seem to indicate there is a problem. I have checked /var/log/secure, /var/log/messages, /var/log/audit/audit.log, /var/log/httpd/misp.local*.log, and MISP/app/tmp/logs and have no indication that either SELinux or some aspect of MISP is causing a malfunction. In fact, if your run systemctl status misp-workers.service you'll see that the service is happy and the workers are still happily grinding along.

I have discovered however that if you don't like red text all over the workers tab, if you restart the misp-workers service, (e.g. systemctl restart misp-workers.service) the web UI acknowledges that the workers have been restarted and will register green across the board.

Bottom line is that this issue is entirely beyond me, and will likely take some coordination with the MISP team to resolve properly.

RHEL8 Users, this script is NOT yet supported for use on Redhat 8. The MISP installation instructions indicated that packages for GNUPGP aren't available yet, and the installation section for misp-modules clearly states that it is not working. Additionally, the EPEL repository is not yet available for redhat 8, and according to the fedora project status page, is not projected to be available until RHEL 8.1 is released. On top of ALL that, the RHSCL (Redhat Software Collection) repos aren't a thing in RHEL 8 anymore, meaning that we no longer have to use SCL to set environments correctly to use the SCL packages. This is both good for you, and bad, meaning that significant portions of this script will probably need to be re-done for RHEL8 users when we get to that. Sit tight, as soon as I can, I'll update things here to support RHEL8 users.

Acknowledgements

• Hurricane Labs, for giving me the time and motivation to work on this

• the MISP Project team for being extremely responsive, and providing most of the code and documentation required to actually write this script. For the most part, all I had to do was copy/paste, fix a couple of issues here and there, and build in some error-checking/catching here and there. Many thanks!

• Munin for pointing me to a stackexchange thread on ways to trick systemd into accepting arguments in systemd service files (https://superuser.com/questions/728951/systemd-giving-my-service-multiple-arguments). This helped to resolve an issue with the misp-modules.service systemd service file in the installation documentation : for some reason it wasn't recognizing command line arguments to misp-modules, so we passed those arguments as a variable, and it worked. neat.

Patch Notes:

2019-09-18

• Got a copy of RHEL7 and a developer license to test out whether this script actually works on RHEL7, Things seem to work perfectly fine . I had to make a slight change to how the script handles installing the EPEL repo RPM. If its already installed (e.g. a user already installed it, or the script failed to run, but managed to install it before crashing) , then using yum install will fail because its already there. So if we see yum install fail to install the EPEL repo, the script will attempt to run yum reinstall [epel repo] and if that fails, we'll consider it an error to stop script execution on.
• Also updated the documentation to tell RHEL7 users that you need a subscription with access to the EPEL repos and that the redhat developer portal is a thing.