Talk:Development/Beta Cycle

From MusicBrainz Wiki
Jump to navigationJump to search

Ian's Comments

Beta Flowchart

First of all, I've made a flowchart out of this, because I think multiple views on this process can make it more understandable. Click to enlarge, etc.

I see a few problems:

  1. The start of the process is actually step 6, not step 1.
  2. We should really specify what to do for schema changes, since right now apparently the answer is "uh, guess we go back to the start?".
  3. ... which brings us to another problem, which is that shipping code to test is conditional -- it only happens if code needs to go through code review. This should not be the case; if code doesn't need code review it should be shipped to both beta and test, so that test doesn't miss out on changes. This also applies to code that isn't on test during the code-review period, but rather mbsandbox or such -- it should end up on test so that people who still want to test those changes, *with* other changes, but *without* using live data, can.

Therefore, my proposal for changing this would be:

  1. Renumber 6 to 1, and increment steps 1-5.
  2. Add "ship to test, if the change isn't already there" to the "Ship code to beta, change ticket to In Beta Testing" step.
  3. Come up with something for schema changes -- I'd propose shipping to test but not beta (i.e., the same path, but skip the actual "ship to beta" part, since that's impossible for such changes), unless we want to institute a schema change beta test server (but I see this as silly; we have test and that seems like a very reasonable place to put these changes).

Ianmcorvidae 00:46, 12 June 2012 (UTC)

  1. I disagree with step 6 being the start. step 6 is involves checking for feedback on a fix you shipped to beta, before you can get such feedback you need to have shipped something to beta, so naturally, step 6 cannot be the first step.
  2. -
  3. I do not think it is particularly important that code is shipped to both test and beta. In general my expectation is that testers actively use beta (because their work would be wasted on test), and only go to test when someone asks them to test a particular thing shipped to test. I think it is fine for the integration testing to only happen on beta. Obviously if a tester would like to test something on test which is only on beta, they should simple ask the developer to ship that particular feature to test as well, and the developer should do so in a timely manner :).


  1. No -- step 6 involves checking if there are tickets for beta (that apply to you). When you haven't yet shipped anything, the answer is trivially no and you move to step 1. The main thing that concerns me is that keeping it as step 6 highlights writing new code, when the focus should be on fixing anything that's already in beta, or taking it out of beta if it can't be fixed before the next release. The core of the problem, of course, is that this process combines the flow for an individual ticket with the developer decision-making process (a view that considers all tickets). In any case, it's a nitpick, so I'm not too concerned so long as we follow what I think we already agree on.
  2. -
  3. Shipping to beta only seems fine, generally, but it doesn't deal with schema changes and new edit types and such -- shipping to test does. Perhaps we can compromise to not require ship-to-test for most changes, but to require it for anything that can't go to beta?

Ianmcorvidae 00:36, 28 June 2012 (UTC)