ID: TM_01.01 Charter ----------------------------------- To investigate and test the 'Sort' functionality of TradeMe's search results and Category listings Tester ----------------------------------- Aaron Hodder Setup ----------------------------------- Firefox 27.0 Windows 8 http://www.trademe.co.nz Start Time ----------------------------------- 10:15am 13/02/2014 Paused: 10:45am Resume: 11:45am Paused: 12:32pm Resume: 1:42pm End Time ----------------------------------- 2:36pm Approximate time spent on Charter ----------------------------------- 2h:10min Test Notes ----------------------------------- The Sort options are: *Featured first *Lowest price *Highest price *Lowest buy now *Highest buy now *Most bids *Latest listings *Closing soon *Title Using the "Antiques & collectables" category: -Performed visual verification that the filter controls were consistently rendered for each sort option -Created visual coverage model for the combinations of Sort options vs Sort options Sympathetic testing of each sort option in turn: Featured first: =============== Noted first 198 listings displayed (at 10:36am) were all 'Featured' A sample of the following search result pages all had non-featured listings. (Given access to a test database we would perform a more controlled test with known listings to determine the correctness of this functionality) Determined that the secondary sort appears to be by time until auction closes. That is, after sorting by Featured vs NonFeatured, auctions that close sooner are listed higher Closing soon ============= Visual verification of first 2 pages, Page 20, and the last page indicates this function sorts the listings from auctions closing soonest, to auctions closing latest. The soonest to close listing observed was closing in 1 second and the last to close closed on "Wed 19 Feb, 2:00 pm". If we had access to the database we could determine if ALL listings in the category are displayed, or whether there is a cut off closing time after which listings are not displayed. Note: Appears only 32000 listings display Question: How is the "Closing soon" sort secondarily sorted. Answer: I believe the granularity of the 'closing soon' sort is down to the second. I am assuming that there would be a timestamp of when it was written into the database that determines the listing, thus, not requiring a secondary sort. Would follow up this question with a developer Lowest Price ============= Noting that a 'Buy Now' price and a 'Bid' price is displayed. Appears to be sorted by the 'Bid' price. See screenshot "Lowest price using bid.png" This can lead to situations where a lower 'buy now'can appear after a higher 'buy now' where there is no reserve and no bids, but both listings have different minimum bids. For an example, see Screenshot 001. Based on how the 'Highest Price' sort works (see below) I now wonder if it is the lower of the 'Buy now' price and the 'Bid' price. I cannot determine how the secondary sort works. It appears to be a combination of time until close, presence of a reserve, presence of bids, and the buy now. More time can be spent determining this. Access to test data would also help. Follow up question to dev Highest Price ============= Appears to be sorted by the higher of the 'Buy now' and the 'bid' price. See screenshot "Highest price using buy now.png" As with 'Lowest Price' I cannot determine how the secondary sort works. Follow up with developer. Lowest Buy Now price ==================== Appears to sort by buy now price in ascending order. Listings with no Buy Now price are not displayed. (compared listings displayed in Lowest Price display vs Lowest Buy Now price display to determine this) I cannot determine the secondary sort order. Please see screenshot "lowest buy now secondary.png". You will note that listing 1 has a bid higher than listing two. You will note that listing 5 closes sooner than listing 4. Will follow up with PM or Developer. Highest Buy Now price ===================== Appears to sort by Buy Now price in descending order. Listings with no Buy Now price are not displayed. (compared listings displayed in Lowest Price display vs Lowest Buy Now price display to determine this) I cannot determine the secondary sort order. Will follow up with PM or Developer. Most Bids ========= Appears to display the listings in descending order of most bids. The top 32000 listings are returned. Visually verified page 1, page 10 and page 160 Cannot determine secondary order. Latest Listings =============== Without the ability to control the adding of listings this cannot be determined with a high degree of confidence. Also, the listings themselves don't provide a 'listed on' time stamp. However, each listing as a unique id, eg: "Listing #: 696200821". The top three listings had ids of: 696200821 696200741 696200685 ASSUMING that ids are allocated in sequential numerical order, then it appears that the listings are displayed as 'Newest first'. While testing, the page was refreshed several times, and new listings did appear at the beginning of the results indicating that new listings were being added. Further testing in a controlled test environment would be required to determine with more confidence the behaviour of this function. Title ============ The first 15 listings do not appear to follow any logical sort order. Defect TM001 logged. Furthermore, I'm seeing further anomalies: eg, page 500 has listings starting with 'H (upper and lower)' (http://www.trademe.co.nz/browse/categorylistings.aspx?page=500&rptpath=124-&mcatpath=electronics-photography&sort_order=title_asc) but page 2 has 'i's (http://www.trademe.co.nz/browse/categorylistings.aspx?page=2&rptpath=124-&mcatpath=electronics-photography&sort_order=title_asc) Stopping heuristic: "We know that things are going to be modified so much that any more testing will be invalidated by the changes." Questions ------------------------------------ Are the 'Sort' options all present and in the order desired? What are the secondary sort functions for all the sort types? Is there a specification for how the title sort should be ordered? Issues ------------------------------------ Testing in production means we don't have access to the database to verify our observations. We also don't have control over the test data which may affect our ability to observe potential defects. Bugs logged ------------------------------------ TM001: Anomaly with 'Title' sort order [hyperlink] (Case 30190 raised) TM002: Further anomalies with the 'Title' search results Follow up Charters ------------------------------------ -Investigate and test filter options -Create Combinatorial tests for the Sort options vs the Filter options. Supplementary material -------------------------------------