Skip to content

Latest commit

 

History

History
93 lines (81 loc) · 3.61 KB

TODO.md

File metadata and controls

93 lines (81 loc) · 3.61 KB
Raw

NATL (Nitrogen Automation Test Lab)


TODO

Dashboard

  • P1Charts
    • Pass rate of last several executions
    • Coverage of all test cases(add manual test cases field to project table)

  • Navigation

    • Replace current project preference <select> with a metro dropdown menu

  • Execution

    • Add/remove entries
    • P1Display result
    • P1Rerun all/failed, will create a new execution
    • Rerun selected
    • Automatically rerun(replace former entries on webpage, but keep data)
    • P1Pie chart to display pass rate
    • Append to current execution?
    • P1Queue entry count, execution's created date on execution list page
    • Highlight failed cases on suite/execution detail page or, support sort function on those pages

  • TestCase
    • P1Display latest result
    • P1Run selected(in a new execution or append to existing ones)
    • Filter(testGroup/execResult/execution)
    • Historical results, and filter(env/browser/slave)
    • Group test cases by failure message
    • P2Treeview to display test cases: left-right panel, left panel to display treeview, right to display test case detail (result history, linked defects) http://codepen.io/fsbdev/embed/nILhu?height=600&user=fsbdev&type=result&slug-hash=nILhu&safe=true&default-tab=result&animations=run#js-box
    • Gridview to display test cases: add/remove columns, filter(with condition AND/OR), group by column

  • TestGroup
    • Historical pass rate

  • Project
    • Project concurrency
    • Reload: reload all TestCase/TestGroup/TestSuite, call Jenkins job to do this, add a process bar on the webpage
    • Project level execution parameters, user can decide whether to inherit them when creating test suite

  • TestResultDetail
    • Failure reason(regression/environment/scriptError)
    • Comments
    • Log(provided by TestPartner)(redirect TestNG commandline output to a file, then save to DB)

  • TestSuite
    • Single/multiple TestGroup/TestCase with default paramters, user can use default parameter or define their own when creating executions
    • Reload automatically when reloading all test cases
    • Historical pass rate (need to create relationship with execution)
    • Define test suite based on certain condition: 1) TestGroup.name == "ABC" OR "DEF"
    • BUG - test suite details page doesn't show last execution results
    • Execution history list, list all executions triggered by current test suite.

  • Slave
    • P2Project priority

  • P2Schedule

Daemon

  • P1Deployment(stop/deploy/start)
  • Run TestNG programatically inside net.nitrogen.ates.TestRunner, to fit @Parameters requirement
  • Timeout for queue entry (to avoid 'dead' queue entry to block slave), try to use apache.commons.exec DefaultExecuteResultHandler.waitFor(long timeout)

Miscellaneous

  • Renaming:Round -> Execution
  • Move all ates projects into one git repo
  • Deploy ATES on VMs in Shanghai
  • Multiple browser support(IE10/Firefox/Chrome)
  • Ant support
  • Test method dependency
  • P2Email notification against execution
  • P1Hide TestResult page
  • Test case mapping with test case management system
  • P1Migration manual
  • P1Modify Jenkins configuration to: copy maven resources to project/{projectId}/main/resources (clean before copying)
  • P1Let users define Java packages that contain valid tests they want to import, then run TestNG programatically to import tests with a virtual TestNG suite. This way users don't have to add ATESTestImporter.xml to their project.
  • Add DB build
  • BUG: some failures are noted as unknown
  • Exception in BeforeClass method not caught