Skip to content

insights: regression testing for settings based insights for 3.34

Created by: coury-clark

Frontend insights (as necessary)

  1. I can create an insight from my default insight landing page and it gets saved to the page @coury-clark
    1. I can then add this insight to an org visible dashboard and return to view it there
    2. I can add this insight to a second, global visible dashboard and return to view it there
    3. I can remove this insight from the first org visible dashboard and return to verify it's not there
    4. I can return to the second (global) dashboard to verify that it is still present there
    5. I can add this insight to a private dashboard of mine
    6. I can return to the global dashboard, verify the insight is still there, then remove this insight from the global dashboard
    7. I can return to the private dashboard to still view this insight
    8. I can delete this insight
  2. I can create an insight from an org dashboard and it is saved to that same org dashboard
  3. I can create an insight from a global dashboard and it is saved to a global dashboard
  4. I can create an insight from a private dashboard and it is saved to that private dashboard
  5. I can create an insight, save it to an org dashboard, add it to a private dashboard, change the title of the insight on the org dashboard, and verify the title changed on the private dashboard as well
  6. I can create an insight, save it to an org dashboard, add a copy of it to the same org dashboard, change the title of the copy, and confirm that the title does not change on the original
  7. I can create an insight from a private dashboard, add the insight to an org dashboard, and then verify that a second user in the org can see the insight on the org dashboard
  8. I can verify that a second user in my org cannot see the names/ability to view my private dashboards
  9. I can verify a user on the instance cannot see the names/ability to view an org dashboard for an org they are not a part of
  10. I can create two users with different repo permissions, create an insight over "all repos" that should be different for the two users (so it involves matching on a repo one of the users does not have access too), and then view the insight from both users and verify the values are actually different.
  11. I can save default filters to an insight on an org dashboard, add that insight to a private dashboard, view it on the private dashboard, and verify the default filters persisted.
  12. I can create a copy of an insight on a dashboard, save new default filters to the original, and verify the default filters do not apply to the copy.

Pings:

  1. I can create a new insight and see that it shows up in our pings as increasing the total # of insights created
  2. We created a separate ping to track the total # of insights in existence (beyond just the events of additions + removals)
  3. I can create a new user and create a new insight and then see it shows up in pings as a new "first time creator" creating an insight
  4. I can verify the total count of "org visible insights" (any insight on an org or global dashboard) shows up in pings, and each insight is counted only 1 time.
  5. Removing an insight shows up in pings as an insight removal
  6. All dashboard pages of any visibility still log pageviews in the pageview pings
  7. Any new "save / save as a copy" still show up in our button click pings for the "submit on creation form" button event.

Data Migration specific:

  1. Frontend insights created before the migration continue to exist on their dashboards, including: 2. Data series' labels and colors are respectively the same 3. The titles are the same 4. The repositories it runs over are the same
  2. Backend insights created before the migration continue to exist on their dashboards, including: 2. Data series' labels and colors are respectively the same 3. The titles are the same 4. The repositories it runs over are the same 5. Default filters applied still persist (may need to create some specifically to test this)
  3. Insights that existed only my "all insights" dashboard for either private or org space are migrated to a (no longer automatic) [private / org] insight's dashboard

Backend insights (as necessary)

  1. I can create an insight from my default insight landing page and it gets saved to the page
    1. I can then add this insight to an org visible dashboard and return to view it there
    2. I can add this insight to a second, global visible dashboard and return to view it there
    3. I can remove this insight from the first org visible dashboard and return to verify it's not there
    4. I can return to the second (global) dashboard to verify that it is still present there
    5. I can add this insight to a private dashboard of mine
    6. I can return to the global dashboard, verify the insight is still there, then remove this insight from the global dashboard
    7. I can return to the private dashboard to still view this insight
    8. I can delete this insight
  2. I can create an insight from an org dashboard and it is saved to that same org dashboard @coury-clark
  3. I can create an insight from a global dashboard and it is saved to a global dashboard
  4. I can create an insight from a private dashboard and it is saved to that private dashboard
  5. I can create an insight, save it to an org dashboard, add it to a private dashboard, change the title of the insight on the org dashboard, and verify the title changed on the private dashboard as well
  6. I can create an insight, save it to an org dashboard, add a copy of it to the same org dashboard, change the title of the copy, and confirm that the title does not change on the original
  7. I can create an insight from a private dashboard, add the insight to an org dashboard, and then verify that a second user in the org can see the insight on the org dashboard
  8. I can verify that a second user in my org cannot see the names/ability to view my private dashboards
  9. I can verify a user on the instance cannot see the names/ability to view an org dashboard for an org they are not a part of
  10. I can create two users with different repo permissions, create an insight over "all repos" that should be different for the two users (so it involves matching on a repo one of the users does not have access too), and then view the insight from both users and verify the values are actually different.
  11. I can save default filters to an insight on an org dashboard, add that insight to a private dashboard, view it on the private dashboard, and verify the default filters persisted.
  12. I can create a copy of an insight on a dashboard, save new default filters to the original, and verify the default filters do not apply to the copy.

Pings:

  1. I can create a new insight and see that it shows up in our pings as increasing the total # of insights created
  2. We created a separate ping to track the total # of insights in existence (beyond just the events of additions + removals)
  3. I can create a new user and create a new insight and then see it shows up in pings as a new "first time creator" creating an insight
  4. I can verify the total count of "org visible insights" (any insight on an org or global dashboard) shows up in pings, and each insight is counted only 1 time.
  5. Removing an insight shows up in pings as an insight removal
  6. All dashboard pages of any visibility still log pageviews in the pageview pings
  7. Any new "save / save as a copy" still show up in our button click pings for the "submit on creation form" button event.

Data Migration specific:

  1. Frontend insights created before the migration continue to exist on their dashboards, including: 2. Data series' labels and colors are respectively the same 3. The titles are the same 4. The repositories it runs over are the same
  2. Backend insights created before the migration continue to exist on their dashboards, including: 2. Data series' labels and colors are respectively the same 3. The titles are the same 4. The repositories it runs over are the same 5. Default filters applied still persist (may need to create some specifically to test this)
  3. Insights that existed only my "all insights" dashboard for either private or org space are migrated to a (no longer automatic) [private / org] insight's dashboard