ufo-management-server

Management Server component of uProxy for Orgs (UfO)

Status

Tools

UfO is built using the following tools:

• Python as the primary language we code in.
• Google AppEngine for basic server functionality on Google infrastructure.
• Polymer for UI.
• Travis for continuous integration.
• Landscape for code health monitoring.

To manage dependencies we use:

• Setup Script to install various python packages, libraries, and appengine as well as configure the development environemnt.
• Bower to install libraries that we use in the UI (specified in bower.json) including Polymer.

Development Setup

For a high level technical overview of UfO, see the UfO Design Doc.

Prerequisites to Build UfO

Note: you will either need to run these as root, or set the directories they modify (/usr/local) to being editable by your user (sudo chown -R $USER /usr/local)

• git

  • Most machines will have git pre-installed. If you need to install git, you can find instructions from the git website.
  • For Windows, install the desktop git app, which provides an easy-to-use interface for git.

• python

  • Most OSX or Linux machines will have python pre-installed. If you need to install python, you can find instructions from the python website. We develop in python version 2.7 (2.7.10 is the latest stable release at the time of writing).

• pip

  • Pip is a package manager for python which we use to install dependencies. If you need to install pip, you can find instructions from the pip website. Note: future versions of the setup script will likely install this for you, but is has not yet been implemented.

Getting UfO Source and Dependencies

Setup without Setup.sh

1. In your terminal, navigate to a directory where you would like to download UfO. E.g., cd ~/UfO

2. Clone the UfO repository: git clone https://github.com/uProxy/ufo-management-server.git or git clone [email protected]:uProxy/ufo-management-server.git if you have your ssh access to GitHub set up (useful if you use 2-step auth for GitHub, which you should do).

3. Navigate into UfO's root directory with cd ufo-management-server

4. Setup build tools and third-party dependencies, run ./setup.sh setup

• In environments where install permissions may be an issue, it is recommended to run this command as root so that it works cleanly the first time, such as sudo ./setup.sh setup

Setup with Setup.sh

If you already have the setup.sh script but do not have the rest of the code base, this flow is much simpler.

1. Run ./setup.sh install

• In environments where install permissions may be an issue, it is recommended to run this command as root so that it works cleanly the first time, such as sudo ./setup.sh install

Note About Dependency Changes

Note that if any local dependencies have changed (i.e. changes to bower dependencies, updates to pip packages, releases of appengine), you will have to add the new dependency manually or reset completely with the script. To reset with the script, perform the following:

1. Run ./setup.sh clean
2. Run ./setup.sh setup

• In environments where install and delete permissions may be an issue, it is recommended to run both of these commands as root so that it works cleanly the first time, such as sudo ./setup.sh clean and sudo ./setup.sh setup

Setting Up OAuth

As part of the OAuth flow, the server must present a “secret” in order to prove it is authorized to use the APIs that are associated with this secret. As the super administrator, do the following:

1. Go to the Google Developer Console.
2. Create a project. Example name can be “my-management-server”.
3. Select the new project. You may need to refresh the page for the new project to appear.
4. Click to expand “APIs & auth” in the sidebar.
5. Click on “APIs” in the sidebar, and search for Admin SDK in the search box.
6. Click on Admin SDK in the search result. Click on “Enable API” button.
7. Click on “Credentials” in the sidebar.
8. Click on “OAuth consent screen” in the top tab. The only required field is “Product name”. Fill it in and click “Save”.
9. Select “Add credentials” drop-down menu. Select “OAuth 2.0 client ID”.
10. Select “Web application” for Application type.

• It’s okay to leave the default name, or enter something more specific to identify who will be using this secret.

1. Specify the “Authorized JavaScript origins”.

• https://.appspot.com
• http://localhost:<port_number> (for development purpose, double-check if this needs to be fully-qualified hostname)

1. Specify the “Authorized redirect URIs”

• https://.appspot.com/oauth2callback
• http://localhost:9999/oauth2callback (for development purpose, double-check if this needs to be fully-qualified hostname)

1. Click on “Create” button.
2. Click on the subsequent “OK” button.

• You can still access the oauth client id/secret later on for server configuration.

Testing

Our tests run on Travis for every commit, push, and pull request automatically. To execute the tests locally before commit, perform the following:

1. Ensure your environment is configured properly. Run ./setup.sh setup if not.
2. Execute the tests with your modified code.

• Run python -m unittest discover -p "*_test.py"

We have functional UI tests using Selenium Webdriver. Because these tests exercise the actual server in a browser, you will need to point it to an actual server instance and use an actual user account:

1. Ensure you have a dedicated server instance for testing. e.g. my-server-staging.appspot.com
2. Ensure you have a dedicated user account that is configured with domain admin access.
3. Push your modified code to your test instance.
4. Run the tests:

• Navigate to cd tests/ui from the main application directory.
• Run python ui_test_suite.py --server_url='https://my-server-staging.appspot.com' --email='[email protected]' --password='foobar'

Running the Server

Deploying Local Server

Once you have the source code and dependencies in your environment, running the code on your local machine should be very straightforward.

1. Ensure your environment is configured properly. Run ./setup.sh setup if not.
2. Deploy the server locally:

• Run <path_to_appengine_sdk>/dev_appserver.py --port=9999 . Note the trailing period is required.
• This is typically ../google_appengine/dev_appserver.py --port=9999 .

1. The local server can be reached at https://localhost:9999 .
2. The admin console can be reached at https://localhost:8000 by default.

Deploying Cloud Server (Appspot)

1. Ensure your environment is configured properly. Run ./setup.sh setup if not.
2. Deploy the server to the cloud:

• Deploy directly:
  • Run <path_to_appengine_sdk>/appcfg.py -A <your-project-name-here> update . Note the trailing period is required.
  • This is typically ../google_appengine/appcfg.py -A my-management-server update .
• Alternatively, the setup script supports the deploying to a test project.
  • setup.sh deploy

1. The cloud server can be reached at https://.appspot.com/ .
2. The admin console can be reached at https://console.developers.google.com/apis/library?project= .

Setting Up the Server After Deployment

Once the server is deployed, the OAuth client id and secret must be setup and users must be added in order to use the server effectively.

To setup OAuth, perform the following:

1. Have an admin setup your client id and secret in the admin console.
2. Get your client id and secret in the Google Developer Console by visiting my-management-server project > APIs & auth heading > Credentials page > my-web-client as an admin.
3. Copy the Client id and secret from Google Developers Console.
4. Run the setup oauth handler (/setup/oauthclient) before visiting the main page.

• For local deployment, go to https://localhost:9999/setup/oauthclient
• For cloud deployment, go to https://.appspot.com/setup/oauthclient

1. Enter your credentials for the OAuth flow (username and password).
2. Click Allow to allow the server access.
   • It should display a message of ‘Setup your client_id and client_secret in the datastore.’
3. Paste in the values copied from Google Developers Console.
4. Click Submit. This will automatically redirect you to the add user flow in the next subsection.

To add users, perform the following:

1. Run the add users handler (/user/add).
   • For local deployment, go to https://localhost:9999/user/add
   • For cloud deployment, go to https://.appspot.com/user/add
2. If prompted, enter your credentials for the OAuth flow (username and password) and click Allow.
3. From here, users can be added via selection from all users in the domain or from the users within a specific group.
   • To get the users within the domain, click Fetch All Users in Domain.
   • To get the users within a specific group, input a group key (group email alias) into the input box and click Fetch Users From Group.
4. Once the users within the domain or group are displayed, check the box next to those wished to be added.
5. Click Add Selected Users to add those users and generate each's token.
   • Note: there is currently no validation of what users are already added to the management server vs what users are attempted to be added. However, re-adding an existing user will only generate a new token for that user and will not create a duplicate entry.

That's it. You should now be able to add more users if desired or start adding proxy servers. The navigation bar should assist in traversing between handlers and back to the main page. Thanks for using UfO!