Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please check my install.sh version; error catching #51

Open
wants to merge 71 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
cf5258b
adding '|| exit' to essential parts of scripts to
n0s3y Sep 21, 2022
143e0e6
Update install.sh
n0s3y Sep 21, 2022
a9f5007
Update install.sh
n0s3y Sep 21, 2022
5323ad4
Update install.sh
n0s3y Sep 21, 2022
0c554ed
Update install.sh
n0s3y Sep 21, 2022
0adc244
Update install.sh
n0s3y Sep 21, 2022
3469b2a
Update install.sh
n0s3y Sep 21, 2022
02dbb8d
Update install.sh
n0s3y Sep 21, 2022
9b36614
Update install.sh
n0s3y Sep 21, 2022
5baf912
Update README.md
n0s3y Sep 21, 2022
7435589
Update README.md
n0s3y Sep 21, 2022
eaf7296
Update README.md
n0s3y Sep 21, 2022
7f0d0ab
Update README.md
n0s3y Sep 21, 2022
b62a8f5
Update install.sh
n0s3y Sep 26, 2022
4cda507
Create getfirmware.sh
n0s3y Sep 27, 2022
e890b16
Update getfirmware.sh
n0s3y Sep 27, 2022
d7ff353
Update getfirmware.sh
n0s3y Sep 27, 2022
490a473
Update install.sh
n0s3y Sep 27, 2022
d4a3ac1
Update install.sh
n0s3y Sep 27, 2022
27f59e6
Update download.sh
n0s3y Sep 27, 2022
428bf33
Update install.sh
n0s3y Sep 27, 2022
8a6a62d
Update install.sh
n0s3y Sep 27, 2022
7ff55f2
adding content of getfirmware into download.sh directly
n0s3y Sep 27, 2022
1f04964
Merge pull request #1 from hogeschoolnovi/master
n0s3y Sep 27, 2022
99e259d
Update download.sh
n0s3y Sep 27, 2022
e57593a
Update download.sh
n0s3y Sep 27, 2022
94deebf
Update debug.py
n0s3y Sep 28, 2022
a9d332f
Update debug.py
n0s3y Sep 29, 2022
490793f
Delete getfirmware.sh
n0s3y Oct 2, 2022
d77ad95
Merge branch 'pr0v3rbs:master' into master
n0s3y Oct 2, 2022
58fb89c
Update README.md
n0s3y Oct 2, 2022
e39110f
Update README.md
n0s3y Oct 2, 2022
13a072c
Delete README.md
n0s3y Oct 2, 2022
c04013f
Add files via upload
n0s3y Oct 2, 2022
0e7ffb7
Update README.md
n0s3y Oct 2, 2022
de6a145
Update debug.py
n0s3y Oct 7, 2022
1e3c878
Add files via upload
n0s3y Oct 8, 2022
afc2bdc
Add files via upload
n0s3y Oct 8, 2022
bf51066
Update firm.sh
n0s3y Oct 8, 2022
6ceaa1b
Update README.md
n0s3y Oct 8, 2022
90e3871
Update README.md
n0s3y Oct 8, 2022
f347eab
Update firm.sh
n0s3y Oct 8, 2022
a2a627b
Update firm.sh
n0s3y Oct 8, 2022
a05e256
Update runner.sh
n0s3y Oct 8, 2022
c35ed06
Update firm.sh
n0s3y Oct 8, 2022
30e0632
Update runner.sh
n0s3y Oct 8, 2022
8f183ef
Update README.md
n0s3y Oct 8, 2022
5c033cc
Update README.md
n0s3y Oct 9, 2022
f7f8d9b
Update README.md
n0s3y Oct 9, 2022
d7d2e82
Hi There.....
n0s3y Oct 19, 2022
1e9be2e
Update README.md
n0s3y Oct 19, 2022
43a89b7
Update README.md
n0s3y Oct 19, 2022
ab1a2a5
Update debug.py
n0s3y Oct 24, 2022
88a2fd2
Update README.md
n0s3y Oct 24, 2022
a90f021
Update README.md
n0s3y Oct 26, 2022
de5d013
x-terminal-emulator addition to open commands in new terminal
n0s3y Nov 1, 2022
77756af
Delete DIR895LA1_FW113b03.bin.3
n0s3y Nov 1, 2022
d8e9b63
Delete DIR895LA1_FW113b03.bin.1
n0s3y Nov 1, 2022
eb09d7a
Delete DIR895LA1_FW113b03.bin.2
n0s3y Nov 1, 2022
cdb85d0
Delete wget-log
n0s3y Nov 1, 2022
8122dda
cleanup
n0s3y Nov 1, 2022
0103314
Update runner.sh
n0s3y Nov 1, 2022
33d6154
Update firm.sh
n0s3y Nov 1, 2022
15eeef5
Update firm.sh
n0s3y Nov 1, 2022
de40a16
Update firm.sh
n0s3y Nov 1, 2022
cd7590d
Delete binwalk-2.3.3 directory
n0s3y Nov 1, 2022
04c78ae
Update runner.sh
n0s3y Nov 1, 2022
608e5ac
Update runner.sh
n0s3y Nov 1, 2022
e0d0f1f
Test123
n0s3y Nov 13, 2022
097f310
Update README.md
n0s3y Nov 25, 2022
40430d5
Update README.md
n0s3y Nov 25, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 18 additions & 138 deletions README.md
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,157 +1,37 @@
# FirmAE
# FirmAE - User friendly and Error catching
This version of FirmAE has automated installation, GUI firmware debugging and error catching.
For dev; see ./firm.sh, runner.sh for the script chain.

FirmAE is a fully-automated framework that performs emulation and vulnerability analysis. FirmAE significantly increases the emulation success rate (From [Firmadyne](https://github.com/firmadyne/firmadyne)'s 16.28% to 79.36%) with five arbitration techniques. We tested FirmAE on 1,124 wireless-router and IP-camera firmware images from top eight vendors.

We also developed a dynamic analysis tool for 0-day discovery, which infers web service information based on the filesystem and kernel logs of target firmware.
By running our tool on the succesfully emulation firmware images, we discovered 12 new 0-days which affect 23 devices.

# Installation

Note that we tested FirmAE on Ubuntu 18.04.

1. Clone `FirmAE`
```console
$ git clone --recursive https://github.com/pr0v3rbs/FirmAE
```

2. Run `download.sh` script.
```console
$ ./download.sh
```

3. Run `install.sh` script.
```console
$ ./install.sh
```

# Usage

1. Execute `init.sh` script.
```console
$ ./init.sh
```

2. Prepare a firmware.
```console
$ wget ftp://ftp.dlink.eu/Products/dir/dir-868l/driver_software/DIR-868L_fw_revB_2-05b02_eu_multi_20161117.zip
```

3. Check emulation
```console
$ sudo ./run.sh -c <brand> <firmware>
```

4. Analyze the target firmware
* Analysis mode uses the FirmAE analyzer
```console
$ sudo ./run.sh -a <brand> <firmware>
```
## The installation is done by first RECURSIVELY cloning this repo. After that you cd into the directory and run the 'firm.sh' script which will do all the work for you. When this script is done (after about 3-6 minutes), you will be greeted by a pop-up to start running a test emulation. In the type field type: 'dlink' and select the file in the FirmAE folder called: "DIR895LA1_FW113b03.bin".

* Run mode helps to test web service or execute custom analyzer
```console
$ sudo ./run.sh -r <brand> <firmware>
```

## Debug

After `run.sh -c` finished.

1. User-level basic debugging utility. (Useful when an emulated firmware is network reachable)
Standard official Repo information:
_____________________________________________________________________________________________________________
FirmAE is a fully-automated framework that performs emulation and vulnerability analysis. FirmAE significantly increases the emulation success rate (From [Firmadyne](https://github.com/firmadyne/firmadyne)'s 16.28% to 79.36%) with five arbitration techniques. We tested FirmAE on 1,124 wireless-router and IP-camera firmware images from top eight vendors.

```console
$ sudo ./run.sh -d <brand> <firmware>
```
### Installation

2. Kernel-level boot debugging.
Note that we tested FirmAE on Kali 2022.3.

1. Clone `FirmAE`. If you do not use '--recursive', errors will occur.
```console
$ sudo ./run.sh -b <brand> <firmware>
git clone --recursive https://github.com/n0s3y/FirmAE
```

## Turn on/off arbitration

Check the five arbitrations environment variable in the `firmae.config`
```sh
$ head firmae.config
#!/bin/sh

FIRMAE_BOOT=true
FIRMAE_NETWORK=true
FIRMAE_NVRAM=true
FIRMAE_KERNEL=true
FIRMAE_ETC=true

if (${FIRMAE_ETC}); then
TIMEOUT=240
```

## Docker

First, prepare a docker image.
2. Run 'cd FirmAE'
```console
$ sudo ./docker-init.sh
cd FirmAE
```

### Parallel mode

Then, run one of the below commands. ```-ec``` checks only the emulation, and ```-ea``` checks the emulation and analyzes vulnerabilities.
4. Run `firm.sh` script to install FirmAE after cloning. For the installation, a new terminal will open per script that starts. This will be confirmed by a print in the cli saying;'...sh has started'
```console
$ sudo ./docker-helper.py -ec <brand> <firmware>
$ sudo ./docker-helper.py -ea <brand> <firmware>
./firm.sh
```

### Debug mode

After a firmware image successfully emulated.
5. Run `runner.sh` script to run and debug your firmware.
```console
$ sudo ./docker-helper.py -ed <firmware>
```

# Evaluation

## Emulation result

Google spreadsheet -
[view](https://docs.google.com/spreadsheets/d/1dbKxr_WOZ7UmneOogug1Zykj1erpfk-GzRNni8DjroI/edit?usp=sharing)

## Dataset

Google drive -
[download](https://drive.google.com/file/d/1hdm75NVKBvs-eVH9rKb5xfgryNSnsg_8/view?usp=sharing)

# CVEs

- ASUS: [CVE-2019-20082](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2019-20082)
- Belkin: [Belkin01](https://github.com/pr0v3rbs/CVE/tree/master/Belkin01)
- D-Link: [CVE-2018-20114](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2018-20114),
[CVE-2018-19986](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2018-19986%20-%2019990#cve-2018-19986---hnap1setroutersettings),
[CVE-2018-19987](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2018-19986%20-%2019990#cve-2018-19987---hnap1setaccesspointmode),
[CVE-2018-19988](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2018-19986%20-%2019990#cve-2018-19988---hnap1setclientinfodemo),
[CVE-2018-19989](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2018-19986%20-%2019990#cve-2018-19989---hnap1setqossettings),
[CVE-2018-19990](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2018-19986%20-%2019990#cve-2018-19990---hnap1setwifiverifyalpha),
[CVE-2019-6258](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2019-6258),
[CVE-2019-20084](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2019-20084)
- TRENDNet: [CVE-2019-11399](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2019-11399),
[CVE-2019-11400](https://github.com/pr0v3rbs/CVE/tree/master/CVE-2019-11400)

# Authors
This research project has been conducted by [SysSec Lab](https://syssec.kr) at KAIST.
* [Mingeun Kim](https://pr0v3rbs.blogspot.kr/)
* [Dongkwan Kim](https://0xdkay.me/)
* [Eunsoo Kim](https://hahah.kim)
* [Suryeon Kim](#)
* [Yeongjin Jang](https://www.unexploitable.systems/)
* [Yongdae Kim](https://syssec.kaist.ac.kr/~yongdaek/)

# Citation
We would appreciate if you consider citing [our paper](https://syssec.kaist.ac.kr/pub/2020/kim_acsac2020.pdf) when using FirmAE.
```bibtex
@inproceedings{kim:2020:firmae,
author = {Mingeun Kim and Dongkwan Kim and Eunsoo Kim and Suryeon Kim and Yeongjin Jang and Yongdae Kim},
title = {{FirmAE}: Towards Large-Scale Emulation of IoT Firmware for Dynamic Analysis},
booktitle = {Annual Computer Security Applications Conference (ACSAC)},
year = 2020,
month = dec,
address = {Online}
}
./runner.sh
```
A zenity popup to select the .bin firmware file in the FirmAE folder will popup and a type window will ask you to type in the brandname, use 'dlink' for the first test with the provided test firmware.
Empty file.
8 changes: 8 additions & 0 deletions analyses/analyses_log/dlink/1/initializer_time
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Traceback (most recent call last):
File "/home/kali/FirmAE/analyses/./initializer.py", line 6, in <module>
import selenium
ModuleNotFoundError: No module named 'selenium'

real 0m0.040s
user 0m0.021s
sys 0m0.017s
Binary file added analyses/chromedriver
Binary file not shown.
12 changes: 6 additions & 6 deletions debug.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,12 +127,12 @@ def menu():
print('------------------------------')
print('| FirmAE Debugger |')
print('------------------------------')
print('1. connect to socat')
print('2. connect to shell')
print('3. tcpdump')
print('4. run gdbserver')
print('5. file transfer')
print('6. exit')
print('1. Connect to socat')
print('2. Connect to shell')
print('3. TCP-dump')
print('4. Run gdbserver')
print('5. File transfer')
print('6. Exit')

while 1:
menu()
Expand Down
2 changes: 2 additions & 0 deletions download.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ set -e
download(){
wget -N --continue -P./binaries/ $*
}
echo "Downloading firmware..."
wget http://files.dlink.com.au/products/DIR-895L/REV_A/Firmware/Firmware_v1.13b03/DIR895LA1_FW113b03.bin

echo "Downloading binaries..."

Expand Down
63 changes: 63 additions & 0 deletions firm.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#!/bin/bash
# exit when any command fails
set -e
set -o pipefail
# see https://intoli.com/blog/exit-on-errors-in-bash-scripts/ for usage
# keep track of the last executed command
trap 'last_command=$current_command; current_command=$BASH_COMMAND' DEBUG
# echo an error message before exiting
trap 'echo "\"${last_command}\" command filed with exit code $?."' EXIT

abort()
{
echo >&2 '
***************
*** ABORTED ***
***************
'
echo "An error occurred. Exiting..." >&2
exit 1
}

trap 'abort' 0

#CODE
#
#
#
#
whiptail --textbox --scrolltext welcome.txt 10 80

sudo apt update
# If this fails, script should break/exit

# Download git
sudo apt install git

# Clone FirmAE & install it
x-terminal-emulator -e ./download.sh
x-terminal-emulator -e ./install.sh
#
#
#
# End of CODE


# If an error occurs, the abort() function will be called.
#----------------------------------------------------------
# Done!
trap : 0

echo >&2 '
************
*** DONE, FirmAE installed without error codes to be worried about. ***
************
'

# Continuity by starting the emulation script
if zenity --question --title="Start emulation" --text="Do you want to start the emulation?" --no-wrap
then
./runner.sh
else
exit
fi
55 changes: 45 additions & 10 deletions install.sh
Original file line number Diff line number Diff line change
@@ -1,19 +1,43 @@
#!/bin/bash
# exit when any command fails
set -e
set -o pipefail
# see https://intoli.com/blog/exit-on-errors-in-bash-scripts/ for usage
# keep track of the last executed command
trap 'last_command=$current_command; current_command=$BASH_COMMAND' DEBUG
# echo an error message before exiting
trap 'echo "\"${last_command}\" command filed with exit code $?."' EXIT

sudo apt-get update
sudo apt-get install -y curl wget tar git ruby python python3 python3-pip bc
abort()
{
echo >&2 '
***************
*** ABORTED ***
***************
'
echo "An error occurred. Exiting..." >&2
exit 1
}

trap 'abort' 0



sudo apt-get update || exit
sudo apt-get install -y curl wget tar git ruby python3 python3-pip bc || exit
sudo python3 -m pip install --upgrade pip
sudo python3 -m pip install coloredlogs


# for docker
sudo apt-get install -y docker.io
sudo apt-get install -y docker.io

# postgresql
sudo apt-get install -y postgresql
sudo /etc/init.d/postgresql restart
sudo -u postgres bash -c "psql -c \"CREATE USER firmadyne WITH PASSWORD 'firmadyne';\""
sudo -u postgres createdb -O firmadyne firmware
sudo -u postgres psql -d firmware < ./database/schema
sudo -u postgres bash -c "psql -c \"CREATE USER firmadyne WITH PASSWORD 'firmadyne';\"" || true
sudo -u postgres createdb -O firmadyne firmware || true
sudo -u postgres psql -d firmware < ./database/schema || true
echo "listen_addresses = '172.17.0.1,127.0.0.1,localhost'" | sudo -u postgres tee --append /etc/postgresql/*/main/postgresql.conf
echo "host all all 172.17.0.1/24 trust" | sudo -u postgres tee --append /etc/postgresql/*/main/pg_hba.conf

Expand All @@ -29,23 +53,23 @@ wget https://github.com/ReFirmLabs/binwalk/archive/refs/tags/v2.3.3.tar.gz && \
sed -i 's/^install_unstuff//g' deps.sh && \
echo y | ./deps.sh && \
sudo python3 setup.py install
sudo apt-get install -y mtd-utils gzip bzip2 tar arj lhasa p7zip p7zip-full cabextract fusecram cramfsswap squashfs-tools sleuthkit default-jdk cpio lzop lzma srecord zlib1g-dev liblzma-dev liblzo2-dev unzip
sudo apt-get install -y mtd-utils gzip bzip2 tar arj lhasa p7zip p7zip-full cabextract cramfsswap squashfs-tools sleuthkit default-jdk cpio lzop lzma srecord zlib1g-dev liblzma-dev liblzo2-dev unzip

cd - # back to root of project

sudo cp core/unstuff /usr/local/bin/

python3 -m pip install python-lzo cstruct ubi_reader
sudo apt-get install -y python3-magic openjdk-8-jdk unrar
sudo apt-get install -y python3-magic openjdk-11-jdk unrar

# for analyzer, initializer
sudo apt-get install -y python3-bs4
python3 -m pip install selenium
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb; sudo apt-get -fy install
rm google-chrome-stable_current_amd64.deb
python3 -m pip install -r ./analyses/routersploit/requirements.txt
cd ./analyses/routersploit && patch -p1 < ../routersploit_patch && cd -
python3 -m pip install -r ./analyses/routersploit/requirements.txt || true
cd ./analyses/routersploit && patch -p1 < ../routersploit_patch && cd - || true

# for qemu
sudo apt-get install -y qemu-system-arm qemu-system-mips qemu-system-x86 qemu-utils
Expand All @@ -55,3 +79,14 @@ if ! test -e "./analyses/chromedriver"; then
unzip chromedriver_linux64.zip -d ./analyses/
rm -rf chromedriver_linux64.zip
fi

# If an error occurs, the abort() function will be called.
#----------------------------------------------------------
# Done!
trap : 0

echo >&2 '
************
*** DONE, FirmAE installed without error codes to be worried about. ***
************
'
1 change: 1 addition & 0 deletions letsgo.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
You are about to emulate firmware with FirmAE. Things can take a while. Don't panic. Give the script about 10-15 minutes.
21 changes: 21 additions & 0 deletions runner.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash



FILE=$(zenity --file-selection --title="Select a firmware file" --file-filter="*.bin")
if [ -z "$FILE" ]
then
exit 1
fi
BRAND=$(zenity --entry --title="Add brand" --text="Enter name of brand:")
if zenity --question --title="FirmAE firmware runner" --text="Would you like to start the emulation?"

MODE=$(zenity --entry --title="Add mode. Choose between: '-d', '-c', '-a'. See the oficial documentation for more." --text="Enter mode:")


then
./init.sh
sudo ./run.sh ${MODE} ${BRAND} ${FILE}
else
zenity --text-info --filename="$FILE" --title="Firmware runner"
fi
2 changes: 2 additions & 0 deletions welcome.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Welcome to FirmAE Simplified.
This version is maintained by n0s3y. Bugs: if you see something, say something.