diff --git a/README.md b/README.md index 8d8e472..3005acb 100644 --- a/README.md +++ b/README.md @@ -3,16 +3,12 @@ A simple security system to run on a [Raspberry Pi](https://www.raspberrypi.org/). Features: + - Motion detection and photo capture using the camera. - Mobile notifications with photos. - Detects when you are home and arms or disarms automatically. - Can be remotely disabled or queried using [Telegram](https://telegram.org/). -Similar to these products: - - - https://www.kickstarter.com/projects/vivienmuller/ulo/ - - http://canary.is/ - ![rpi-security 1](../master/images/rpi-security-1.jpg?raw=true) ![rpi-security 2](../master/images/rpi-security-2.jpg?raw=true) @@ -21,43 +17,51 @@ Similar to these products: ## Requirements You will need this hardware: - - Raspberry Pi with camera interface. I use a model A+. + + - Raspberry Pi with camera interface. - Raspberry Pi camera module. - - USB Wi-Fi that supports monitor mode. I used a RT5370 based adapter, they are cheap at about €6 and easy to find. - - An enclosure of some sort. Details of the hardware I made is [here](hardware). + - A Wi-Fi adapter that supports monitor mode (see [note](#WiFi-adapter-arrangement)) + +Software requirements: -Other requirements: - - A [Telegram bot](https://telegram.org/blog/bot-revolution). It's free and easy to setup. - - Raspbian distribution installed. I used 9 (stretch). You could possibly use a different OS but I haven't tried it. + - A [Telegram bot](https://core.telegram.org/bots). It's free and easy to setup. + - Raspbian Stretch Lite distribution installed. - Python 3. ## How it works -### Automatic presence detection +#### Automatic presence detection One of my main goals was to have the system completely automatic. I didn't want to have to arm or disarm it when leaving or arriving home. I figured the easiest way to achieve this was to try and detect the mobile phones of the home occupants. Conceptually this was quite simple but in practice it was the most challenging part because: + - Capturing all packets on a Wi-Fi interface is too resource intensive. - There are presently no good 5Ghz USB Wi-Fi adapters that support monitor mode. This means packet monitoring is restricted to 2.4Ghz where most modern mobile phones use 5Ghz now. - Mobile phones are not always online and sending packets over Wi-Fi. Sometimes they stay unconnected for 15 minutes or longer. - Even with 99% accuracy false alarms are annoying. -After much testing I used an approach that mixes active (ARP scan) and passive (packet capture) detection over the Wi-Fi adapter based on knowing the MAC addresses of the mobile phones. The mobile phone MAC addresses are set in the configuration and the rpi-security application captures packets on a monitor mode interface with the following filter: +After much testing I used an approach that mixes active (ARP scan) and passive (packet capture) detection over the Wi-Fi adapter based on knowing the MAC addresses of the mobile phones. The mobile phone MAC addresses are set in the configuration and rpi-security captures packets on a monitor mode interface with the following filter: + 1. Wi-Fi probe requests from any of the configured MACs. 2. Any packets sent from the configured MACs to the host running rpi-security. The application resets a counter when packets are detected and if the counter goes longer than ~10 minutes the system is armed. To eliminate the many false alarms, when transitioning from armed to disarmed state or vice versa, the application performs an ARP scan directed at each of the configured MAC addresses to be sure they are definitely online or offline. Both iOS and Android will respond to this ARP scan 99% of the time where a ICMP ping is quite unreliable. By combining the capture of Wi-Fi probe requests and using ARP scanning, the Wi-Fi frequency doesn't matter because mobile phones send probe requests on both frequencies and ARP scan works across both frequencies too. -### Notifications +#### Motion detection -A [Telegram](https://telegram.org/blog/bot-revolution) bot is used to send notifications with the captured images. They have good mobile applications and a nice API. You can also view the messages in a browser and messages are synced across devices. +Motion detection is done using [OpenCV](https://opencv.org/). Each motion detection will save 4 pictures in `/tmp`: + + - frame.jpg: The picture with rectangles surrounding the motion + - gray.jpg: The picture with grayscale and blur (which will be use to detect motion between the current frame and the previous one) + - abs_diff.jpg: The absolute difference between they grays frames (current and previous) + - thresh.jpg: A threshold has been applied to be sure the motion is important enough to be detected. -If the system is in an armed state and motion is detected then a message with the captured image is sent to you from the Telegram bot. +#### Notifications -Notifications are also sent on any alarm state change. +A [Telegram](https://core.telegram.org/bots) bot is used to send notifications with the captured images. They have good mobile applications and a nice API. You can also view the messages in a browser and messages are synced across devices. If the system is in an armed state and motion is detected then a message with the captured image is sent to you from the Telegram bot. Notifications are also sent on any alarm state change. ![rpi-security 2](../master/images/rpi-security-notification.png?raw=true) -### Remote control +#### Remote control You can send the Telegram bot commands that trigger certain actions. @@ -67,69 +71,73 @@ You can send the Telegram bot commands that trigger certain actions. - */photo*: Captures and sends a photo. - */gif*: Captures and sends a gif. -![rpi-security 4](../master/images/rpi-security-status-message.png?raw=true) +#### Python -### Python +The application is written in python 3 and large parts of the functionality are provided by the following pip packages: -The application is written in python 3. Large parts of the functionality are provided by the following pip modules: - [picamera](https://github.com/waveform80/picamera) - - [scapy](http://www.secdev.org/projects/scapy/) + - [kamene](https://github.com/phaethon/kamene) - [python-telegram-bot](https://github.com/python-telegram-bot/python-telegram-bot) + - [opencv-python](https://github.com/skvark/opencv-python) The application uses multithreading in order to process events asynchronously. There are 4 threads: + - telegram_bot: Responds to commands. - monitor_alarm_state: Arms and disarms the system. - capture_packets: Captures packets from the mobile devices. - process_photos: Sends captured images via Telegram messages. -### Motion detection -Previously, the motion detection was made by the default RpiMotionDetector from RpiCamera which could cause many false positive. -In the newer version, the motion detection is done thanks to OpenCV and Image algorithm, reducing the number of false positive. -Each motion detection will save 4 pictures in /tmp: - - frame.jpg: The picture with rectangles surrounding the motion - - gray.jpg: The picture with grayscale and blur (which will be use to detect motion between the current frame and the previous one) - - abs_diff.jpg: The absolute difference between they grays frames (current and previous) - - thresh.jpg: A threshold has been applied to be sure the motion is important enough to be detected. - ## Installation, configuration and Running -The interface used to connect to your WiFi network must be the same interface that supports monitor mode. And this must be the same WiFi network that the mobile phones connect to. +First ensure your WiFi is [set up correctly](#WiFi-adapter-arrangement)) -First install required packages: +Install required packages: -``` -sudo apt-get update -sudo apt-get install -y tcpdump iw python3-dev python3-pip libjpeg8-dev zlib1g-dev libffi-dev python3-numpy libopenjp2-7-dev libtiff5 install libatlas-base-dev libjasper-dev libqtgui4 python3-pyqt5 libqt4-test -sudo pip3 install --upgrade pip +```console +sudo apt update +sudo apt install -y libhdf5-100 libharfbuzz0b libwebp6 libjasper1 libilmbase12 libopenexr22 libgstreamer1.0-0 libavcodec-extra57 libavformat57 libswscale4 libgtk-3-0 libqtgui4 libqt4-test libatlas-base-dev tcpdump iw python3-dev python3-pip libjpeg8-dev zlib1g-dev libffi-dev python3-numpy libopenjp2-7-dev libtiff5 ``` -Install rpi-security, reload systemd configuration and enable the service: +Install open-cv and rpi-security: +```console +sudo pip3 install opencv-contrib-python opencv-contrib-python-headless +sudo pip3 install --no-binary :all: https://github.com/FutureSharks/rpi-security/archive/1.0.zip ``` -sudo pip3 install --no-binary :all: https://github.com/FutureSharks/rpi-security/archive/master.zip + +Reload systemd configuration and enable the service: + +```console sudo systemctl daemon-reload sudo systemctl enable rpi-security.service ``` -Add your MAC address or addresses, Telegram bot API key and any other changes to ``/etc/rpi-security.conf``. +Add your MAC address or addresses, Telegram bot API key and any other changes to `/etc/rpi-security.conf`. -Ensure you have enabled the camera module using ``raspi-config``. +Ensure you have enabled the camera module using `raspi-config`. And start the service: -``` +```console sudo systemctl start rpi-security.service ``` -You need to send at least one message to the Telegram bot otherwise it won't be able to send you messages. This is so the service can save the telegram chat_id. So just send the ``/status`` command. +You need to send at least one message to the Telegram bot otherwise it won't be able to send you messages. This is so the service can save the telegram chat_id. So just send the `/status` command. -It runs as a service and logs to syslog. To see the logs check ``/var/log/syslog``. +It runs as a service and logs to syslog. To see the logs check `/var/log/syslog`. -There is also a debug option that logs to stdout: +## Debug and troubleshooting -``` +You can start `rpi-security.py` manually with debug output. First add the monitor mode interface: + +```console root@raspberrypi:~# iw phy phy0 interface add mon0 type monitor root@raspberrypi:~# ifconfig mon0 up +``` + +Then start with debug output: + +```console root@raspberrypi:~# rpi-security.py -d 2016-05-28 14:43:30 DEBUG rpi-security.py:73 MainThread State file read: /var/lib/rpi-security/state.yaml 2016-05-28 14:43:30 DEBUG rpi-security.py:44 MainThread Calculated network: 192.168.178.0/24 @@ -144,11 +152,29 @@ root@raspberrypi:~# rpi-security.py -d 2016-05-28 14:44:48 DEBUG rpi-security.py:280 Dummy-1 Motion detected but current_state is: disarmed ``` -This is all that is required on my Raspberry Pi Model A+. +And then delete the interface when complete: -This shows my WLAN network device arrangement: +```console +iw dev mon0 del +``` + +## WiFi adapter arrangement + +Your WLAN or WiFi adapter must support monitor mode. The Raspberry Pi built-in wireless LAN adapters do **not** support monitor mode by default. Currently the only way to get monitor mode working for the built-in WiFi adapters is to use [nexmon](https://github.com/seemoo-lab/nexmon) and this is not simple. + +The easiest way to get a monitor mode WiFi adapter is to just buy a RT5370 based adapter. They are cheap at about €6 and easy to find. + +The interface used to connect to your WiFi network must be the same interface that supports monitor mode. And this must be the same WiFi network that the mobile phones connect to. This is because there is a packet capture running to listen for mobile phone ARP replies and Wi-Fi probe requests. +If you are using a USB adapter and want to disable your onboard WiFi adapter run this and reboot: + +```console +sudo sh -c "echo 'blacklist brcmfmac\nblacklist brcmutil' > /etc/modprobe.d/disable-onboard-wifi.conf" ``` + +This shows a working WiFi adapter arrangement: + +```console root@raspberrypi:~# iw dev phy#0 Interface mon0 @@ -164,18 +190,18 @@ phy#0 channel 1 (2412 MHz), width: 40 MHz, center1: 2422 MHz ``` -You could have interfaces with different names, just be sure to change the ``network_interface`` parameter in ``/etc/rpi-security.conf`` and also the reference to mon0 in [rpi-security.service](https://github.com/FutureSharks/rpi-security/blob/master/etc/rpi-security.service) +You could have interfaces with different names, just be sure to change the `network_interface` parameter in `/etc/rpi-security.conf` and also the reference to `mon0` in [rpi-security.service](https://github.com/FutureSharks/rpi-security/blob/master/etc/rpi-security.service) -### Older version with PIR sensor motion detection +## Older version with PIR sensor motion detection -Currently the camera is used for motion detection. If you want to use the old version with support for a PIR sensor then look at version 0.7: +Currently the camera is used for motion detection. If you want to use the old version with support for a PIR sensor then look at version `0.7`: -https://github.com/FutureSharks/rpi-security/releases/tag/0.7 +https://github.com/FutureSharks/rpi-security/tree/0.7 -### Reboot on connectivity loss +## Reboot on connectivity loss About once every month or two my Raspberry Pi loses the WLAN connection. I created a cron job to check connectivity and reboot if the check fails. -``` +```console echo '*/20 * * * * root /usr/bin/host api.telegram.org > /dev/null 2>1 || (/usr/bin/logger "Rebooting due to connectivity issue"; /sbin/shutdown -r now)' > /etc/cron.d/reboot-on-connection-failure ``` diff --git a/TODO.md b/TODO.md index 81b3ba5..0edfef4 100644 --- a/TODO.md +++ b/TODO.md @@ -5,4 +5,5 @@ - Update photos now that PIR sensor is not required - Tidy up comments - Remove PIR CAD files +- Fig syslog formatting `Dec 22 10:40:02 raspberrypi monitor_alarm_state.py:...` should include service name - Don't save motion detection related photos to the filesystem, [save directly to OpenCV object](https://picamera.readthedocs.io/en/release-0.6/recipes.html#capturing-to-an-opencv-object) diff --git a/bin/rpi-security.py b/bin/rpi-security.py index be04e62..d470d69 100755 --- a/bin/rpi-security.py +++ b/bin/rpi-security.py @@ -48,8 +48,8 @@ def setup_logging(debug_mode, log_to_stdout): try: rpis = rpisec.RpisSecurity(args.config_file, args.data_file) camera = rpisec.RpisCamera(rpis.photo_size, rpis.gif_size, rpis.motion_size, - rpis.camera_vflip, rpis.camera_hflip, rpis.motion_detection_setting, - rpis.camera_capture_length, rpis.camera_mode) + rpis.camera_vflip, rpis.camera_hflip, rpis.camera_capture_length, + rpis.camera_mode) if rpis.debug_mode: logger.handlers[0].setLevel(logging.DEBUG) except Exception as e: diff --git a/etc/rpi-security.conf b/etc/rpi-security.conf index 9e123e8..868d5b5 100644 --- a/etc/rpi-security.conf +++ b/etc/rpi-security.conf @@ -5,12 +5,12 @@ mac_addresses=aa:aa:aa:bb:bb:bb,cc:cc:cc:dd:dd:dd # The Telegram bot token. telegram_bot_token=999999999:xxxxxxx-zzzzzzzzzzzzzzzzz -# The wireless interface in monitor mode -network_interface=mon0 - # The number of people allowed to chat with Telegram bot telegram_users_number=1 +# The wireless interface in monitor mode +network_interface=mon0 + # Debug mode. Setting this to true will send more verbose logging to syslog debug_mode=false @@ -31,7 +31,7 @@ camera_hflip=false # Number of photos to take or number of GIF frames x3 when motion is detected. # Higher than 3 with gif mode will run out of memory on models with 128MB of RAM. -camera_capture_length=3 +camera_capture_length=2 # Image size for photos photo_size=2592x1944 @@ -41,6 +41,3 @@ gif_size=800x600 # Resolution for motion detection motion_size=1280x720 - -# Motion detection settings: motion_magnitude x motion_vectors. Higher of either setting means less sensitivity. Requires some experimentation. -motion_detection_setting=60x17 diff --git a/hardware/README.md b/hardware/README.md index adbb906..6f4ed17 100644 --- a/hardware/README.md +++ b/hardware/README.md @@ -1,6 +1,6 @@ # Raspberry Pi Security System Hardware -I used a [Raspberry Pi Model A+](https://www.raspberrypi.org/products/model-a-plus/) because it's the smallest pi with the camera interface. The frame is made with [Makerbeam](http://www.makerbeam.com/), 4x vertical 66mm beams and 8x horizontal 54mm beams. +I used a [Raspberry Pi 3 Model A+](https://www.raspberrypi.org/products/raspberry-pi-3-model-a-plus/). The frame is made with [Makerbeam](http://www.makerbeam.com/), 4x vertical 66mm beams and 8x horizontal 54mm beams. Raspberry Pi camera mount 3D printed from [Shapeways](https://www.shapeways.com/). @@ -8,8 +8,6 @@ DWG CAD files are in hardware/cad-files or you can see models here on Spaceways: https://www.shapeways.com/product/K6FLBY3SW/raspberry-pi-camera-module-makerbeam-mount -https://www.shapeways.com/product/F4268ZMG5/raspberry-pi-pir-sensor-makerbeam-mount - The camera mount supports both the V1 and V2 camera modules. ## Photos @@ -18,4 +16,8 @@ The camera mount supports both the V1 and V2 camera modules. ![rpi-security 2](../../master/images/rpi-security-2.jpg?raw=true) -![rpi-security 3](../../master/images/rpi-security-hardware-1.jpg?raw=true) +![rpi-security 3](../../master/images/rpi-security-3.jpg?raw=true) + +![rpi-security 4](../../master/images/rpi-security-4.jpg?raw=true) + +![rpi-security 5](../../master/images/makerbeam-camera-bracket.png?raw=true) diff --git a/hardware/cad-files/rpi-PIR-makerbeam-mount.dwg b/hardware/cad-files/rpi-PIR-makerbeam-mount.dwg deleted file mode 100644 index 61516b3..0000000 Binary files a/hardware/cad-files/rpi-PIR-makerbeam-mount.dwg and /dev/null differ diff --git a/hardware/cad-files/rpi-PIR-makerbeam-mount.stl b/hardware/cad-files/rpi-PIR-makerbeam-mount.stl deleted file mode 100644 index 8f0e5cf..0000000 Binary files a/hardware/cad-files/rpi-PIR-makerbeam-mount.stl and /dev/null differ diff --git a/images/makerbeam-camera-bracket.png b/images/makerbeam-camera-bracket.png new file mode 100644 index 0000000..6e8d23c Binary files /dev/null and b/images/makerbeam-camera-bracket.png differ diff --git a/images/rpi-security-1.jpg b/images/rpi-security-1.jpg index 583d543..9b1faf4 100644 Binary files a/images/rpi-security-1.jpg and b/images/rpi-security-1.jpg differ diff --git a/images/rpi-security-2.jpg b/images/rpi-security-2.jpg index 3451160..d05cfb1 100644 Binary files a/images/rpi-security-2.jpg and b/images/rpi-security-2.jpg differ diff --git a/images/rpi-security-3.jpg b/images/rpi-security-3.jpg new file mode 100644 index 0000000..aaf62bd Binary files /dev/null and b/images/rpi-security-3.jpg differ diff --git a/images/rpi-security-4.jpg b/images/rpi-security-4.jpg new file mode 100644 index 0000000..1b27a39 Binary files /dev/null and b/images/rpi-security-4.jpg differ diff --git a/images/rpi-security-hardware-1.jpg b/images/rpi-security-hardware-1.jpg deleted file mode 100644 index b1f0914..0000000 Binary files a/images/rpi-security-hardware-1.jpg and /dev/null differ diff --git a/images/rpi-security-notification.png b/images/rpi-security-notification.png index 1e19062..2a633f8 100644 Binary files a/images/rpi-security-notification.png and b/images/rpi-security-notification.png differ diff --git a/images/rpi-security-status-message.png b/images/rpi-security-status-message.png deleted file mode 100644 index 47901e6..0000000 Binary files a/images/rpi-security-status-message.png and /dev/null differ diff --git a/rpisec/rpis_camera.py b/rpisec/rpis_camera.py index f632644..8197cde 100644 --- a/rpisec/rpis_camera.py +++ b/rpisec/rpis_camera.py @@ -26,7 +26,7 @@ class RpisCamera(object): captues photos and GIFs. ''' def __init__(self, photo_size, gif_size, motion_size, camera_vflip, - camera_hflip, motion_detection_setting, camera_capture_length, + camera_hflip, camera_capture_length, camera_mode): self.photo_size = photo_size self.gif_size = gif_size @@ -34,8 +34,6 @@ def __init__(self, photo_size, gif_size, motion_size, camera_vflip, self.camera_hflip = camera_hflip self.lock = Lock() self.queue = Queue() - self.motion_magnitude = motion_detection_setting[0] - self.motion_vectors = motion_detection_setting[1] self.motion_framerate = 5 self.motion_size = motion_size self.temp_directory = '/var/tmp' @@ -47,7 +45,6 @@ def __init__(self, photo_size, gif_size, motion_size, camera_vflip, self.camera = PiCamera() self.camera.vflip = self.camera_vflip self.camera.hflip = self.camera_hflip - self.camera.led = False except Exception as e: exit_error('Camera module failed to intialise with error {0}'.format(repr(e))) diff --git a/rpisec/rpis_security.py b/rpisec/rpis_security.py index 47607cb..17045ad 100644 --- a/rpisec/rpis_security.py +++ b/rpisec/rpis_security.py @@ -6,12 +6,12 @@ import yaml import logging -logging.getLogger("scapy.runtime").setLevel(logging.ERROR) +logging.getLogger("kamene.runtime").setLevel(logging.ERROR) from configparser import SafeConfigParser from netaddr import IPNetwork from netifaces import ifaddresses -from scapy.all import srp, Ether, ARP +from kamene.all import srp, Ether, ARP from telegram import Bot as TelegramBot from .exit_clean import exit_error from .rpis_state import RpisState @@ -35,9 +35,9 @@ class RpisSecurity(object): 'photo_size': '1024x768', 'gif_size': '1024x768', 'motion_size': '1024x768', - 'motion_detection_setting': '60x10', 'camera_mode': 'gif', - 'camera_capture_length': '3' + 'camera_capture_length': '3', + 'telegram_users_number': '1' } def __init__(self, config_file, data_file): @@ -131,7 +131,6 @@ def _str2bool(v): self.photo_size = tuple([int(x) for x in self.photo_size.split('x')]) self.gif_size = tuple([int(x) for x in self.gif_size.split('x')]) self.motion_size = tuple([int(x) for x in self.motion_size.split('x')]) - self.motion_detection_setting = tuple([int(x) for x in self.motion_detection_setting.split('x')]) self.camera_capture_length = int(self.camera_capture_length) self.camera_mode = self.camera_mode.lower() self.packet_timeout = int(self.packet_timeout) diff --git a/rpisec/threads/capture_packets.py b/rpisec/threads/capture_packets.py index b1b04c8..8308006 100644 --- a/rpisec/threads/capture_packets.py +++ b/rpisec/threads/capture_packets.py @@ -2,10 +2,10 @@ import _thread import logging -from scapy.all import sniff -from scapy.all import conf as scapy_conf -scapy_conf.promisc=0 -scapy_conf.sniff_promisc=0 +from kamene.all import sniff +from kamene.all import conf as kamene_conf +kamene_conf.promisc=0 +kamene_conf.sniff_promisc=0 logger = logging.getLogger() @@ -13,10 +13,10 @@ def capture_packets(rpis): """ - This function uses scapy to sniff packets for our MAC addresses and updates + This function uses kamene to sniff packets for our MAC addresses and updates the alarm state when packets are detected. """ - logging.getLogger("scapy.runtime").setLevel(logging.ERROR) + logging.getLogger("kamene.runtime").setLevel(logging.ERROR) def update_time(packet): packet_mac = set(rpis.mac_addresses) & set([packet[0].addr2, packet[0].addr3]) packet_mac_str = list(packet_mac)[0] @@ -36,5 +36,5 @@ def calculate_filter(mac_addresses): try: sniff(iface=rpis.network_interface, store=0, prn=update_time, filter=calculate_filter(rpis.mac_addresses)) except Exception as e: - logger.error('Scapy failed to sniff packets with error {0}'.format(repr(e))) + logger.error('kamene failed to sniff packets with error {0}'.format(repr(e))) _thread.interrupt_main() diff --git a/setup.py b/setup.py index fdc5c88..7aba37d 100644 --- a/setup.py +++ b/setup.py @@ -2,7 +2,7 @@ setup( name = 'rpi-security', - version = '0.8', + version = '1.0', author = 'Max Williams', author_email = 'futuresharks@gmail.com', url = 'https://github.com/FutureSharks/rpi-security', @@ -22,8 +22,6 @@ install_requires = [ 'python-telegram-bot', 'picamera', - 'RPi.GPIO', - 'opencv-python', 'imutils', 'numpy', 'configparser', @@ -32,8 +30,10 @@ 'netaddr', 'netifaces', 'pyyaml', - 'scapy-python3', - 'Pillow' + 'kamene', + 'Pillow', + 'opencv-contrib-python', + 'opencv-contrib-python-headless', ], classifiers = [ 'Environment :: Console',