How can we improve Tracepot?

Improve issue grouping, or make configurable

So far only two crashes have come in to Tracepot naturally, and they've both been grouped into one "issue" which isn't correct.

The top level exception in the stack trace in these two crashes is different, but I believe Tracepot is trying to be smart and decides which part of the stack trace is relevant for grouping (presumably that is the line which is highlighted in bold in the stack trace, and I'm guessing that is determined as the first trace element which contains the app package name).

The problem with the grouping is that almost all crashes from our app come through the same function (called "caughtError" in the JavaScript interface), as it's a hybrid app with a crosswalk webview.

The issue ID which demonstrates this incorrect grouping is c541872e059b0214.

The ideal solution here in my opinion would be to add a new (numerical) option per app for "Stack trace depth to use for issue grouping", as I suspect this is going to be slightly different for every app.

2 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    Andrew Beveridge shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →
    started  ·  AdminMartin Milesich (Admin, Tracepot) responded  · 

    Hi Andrew,

    We group issues by class name, file name (including the line number) and app version code. We do not use the whole first line of a stack trace anymore because it created lots of issues where only the first line was slightly different but the rest of the stack trace was the same. Usually when an ID of some sort is included that changes every time like a window ID.

    But I understand your problem. You are catching javascript exceptions and re-throwing them as java exceptions. In the current state all happens in the same file with the same class name and therefore is grouped as 1 issue.

    This is not an easy task to solve because I need to take into consideration stack traces from all other apps and there is no clear sign that this is javascript exception so use different grouping. I found some pattern I could probably use but it needs testing.

    UPDATE 2016-01-26: Added a comment about how the problem was solved.

    Martin

    2 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      Signed in as (Sign out)
      Submitting...
      • AdminMartin Milesich (Admin, Tracepot) commented  ·   ·  Flag as inappropriate

        We are not using the first line of a stack trace because of the problem mentioned at https://github.com/ACRA/acra/issues/130 so we opted only for the class name part of that line. This created another problem where custom exception where grouped together because they use the base exception class.

        So the modification to grouping are to keep grouping with class name as it was before with the exception when the class name is java.lang.Exception then use the whole first line of stack trace. Based on my findings the base exception class is used rarely and almost always when developer throw the exception intentionally.

        I understand that every application is different and could use a different approach and custom configuration would solve that. The problem here is per app configuration will add a lot of complexity to collectors. I am not against it and welcome any improvements to grouping but this will require more feedback and a proper planning.

        I will leave this in Started state so anybody can contribute with their proposals to grouping crashes.

      • Andrew Beveridge commented  ·   ·  Flag as inappropriate

        Hi Martin,

        Thanks for your help - as mentioned by email you've made an adjustment to the grouping and it's now working better for our specific app.
        I'd appreciate understanding how you've solved that though, if you're happy to share.

        I personally still believe this is something which should ideally be configurable per app, as I don't believe there is a one-size-fits-all solution to the problem.

        In our case we've already found two exceptions to the assumption that class name, file name, line number and app version code are sufficient to group crashes in a useful way; apps where there are many crashes which are re-thrown from elsewhere, and apps which have multiple ABIs or less obvious app version codes for each release.

        I believe it is likely that other apps will find similar inconvenient grouping issues as Tracepot starts getting users with more complex app configurations - apps which catch crashes in JNI libraries, for example.

        I propose adding a section to the app general settings page to configure how crashes are grouped; allowing the user to select / remove elements to include while deciding whether to group or separate crashes.

        In our case for example, I would go into that hypothetical configuration panel and remove "App version code", as there are 4 version codes (4 ABIs, armv7, arm64, x86, x64) for every 1 app release, which will likely lead to us seeing multiple duplicate issues.
        Additionally our app has a web component which can gets updated separately from the native android app, so a crash caused by a bug in the web component can exist across different native app version codes, even on the same architecture.

        Anyway, this is probably fairly low priority since you've solved the main problem for us and nobody else is reporting this as an issue just now, but I'd love to see it configurable in the future.

      Feedback and Knowledge Base