- Author identifies an assertion to test No UI
- Author designs and creates tests and, when possible, checks that the test behaves as expected in at least one implementation. No UI
- Author submits tests SVN and/or submit-new.html
- Author gets back automated report of what's wrong submit-new-feedback.html
- Author fixes tests to match format submit-new-feedback.html
- Author resubmits tests, they pass submit-new-feedback.html + submit-new-testsuites.html
- Author waits for review submit-new-done.html
- Author receives review comments on 6/9 tests mail + feedback-from-reviewer.html
- Author fixes tests feedback-from-reviewer.html
- Author resubmits tests feedback-from-reviewer.html
- Author receives acknowledgement that tests have been accepted and checked in mail
- If the Reviewer has 'Owner' or 'Peer' status (see Review Process), the Reviewer searches the submittal data base for tests in the 'Accepted' state; if not, or if no 'Accepted' tests were found, Reviewer searches the submittal data base for tests in the 'Resubmitted' or 'Submitted' state and selects a test to review. (He or she cannot review his or her own tests.) peer-review.html
- Reviewer looks for duplicate tests in the set of 'Approved', 'Accepted', and checked-in tests; if found, reject the lesser-quality test as 'Duplicate', or suggest merging the two tests. peer-review-2.html
- Reviewer checks that the test assertion (whether explicit or implied) is justified by the specification. peer-review-2.html
- If the assertion regards functionality specified outside the css module indicated by the 'help' metadata links, reject as 'OutOfScope'.
- If it addresses an ambiguity or open issue within the spec, set status to 'SpecIssue' and ensure that issue has been posted to www-style@w3.org or public-css-testsuite@w3.org and is being tracked appropriately.
- Reviewer checks the test for correctness. (See the CSS Test Review Checklist for details.) If no problems are found, set the status to 'Accepted'. peer-review-2.html
- Otherwise, Reviewer enters comments explaining what changes need to be made and sets the status to 'NeedsWork'. (If the Reviewer has the Author's permission to make changes directly, the Reviewer may also change the test as necessary. In this case, the status is set to 'Resubmitted' and someone else, possibly the Author, must review.) peer-review-2.html + peer-review-resubmit.html
- Approver searches the submission database for tests in the 'Accepted' state.
- Approver either accepts Reviewer's judgement and marks test as 'Approved'–or follows workflow as for Reviewer, above, except passing review results in 'Approved'.
- Approver may suggest a new filename for checkin.
- Check off X tests and give them all the same comment
- Find test by ID, URL, title, etc.
- Handle renaming of test
- Track changes in same stream as comments (every subversion checkin adds a comment pointing to diff and new version)
- Track status of test (needs review, needs work, etc)
- Accept comments
- Be test suite aware (which test belongs to which test suite); note that a test may belong to multiple test suites
- Opt-in to hear comments on tests you reviewed (and accepted but second-level review found problems with?)
- Easy to set up an account
- Bulk submit preserving folder structure
- Should work on all browsers
- Unified login with wiki and Subversion
- meta data presented at top
- merged list of comments / changes (link to diff) sorted by date
- logged in user may change status and comment
- anonymous user may make a comment, filling out either a name field or filling out username+pass to log in (like on LiveJournal)
- list by status
- search on assertion, title, testid, author, help link, who commented
- batch review, list tests, link to test, checkbox, review
- list title, assertion, link to test, link to full report
- only peers can approve
- flag tests with revisions since last comment
- Should allow substring matching to rename/move multiple files