12:16:06 #startmeeting Mer QA meeting 3/5/2012 12:16:06 Meeting started Thu May 3 12:16:06 2012 UTC. The chair is E-P. Information about MeetBot at http://wiki.merproject.org/wiki/Meetings. 12:16:06 Useful Commands: #action #agreed #help #info #idea #link #topic. 12:16:20 #topic Current status 12:16:25 (I think you can feed it the logs later) 12:16:27 (sorry. couldn't resist) 12:16:40 the problem atm is that although copyproj works, it doesn't handle events correctly 12:16:47 so that's still a WIP 12:17:04 I doubt much will happen until post Tizen conf 12:17:24 ok, sounds good 12:18:03 yes, it'll let us make images 12:20:14 anything else? 12:20:40 lets move on 12:20:57 ok 12:21:03 #topic QA process 12:21:46 I draw first draft about the Mer QA, http://wiki.merproject.org/wiki/Quality/Test_processes#Mer_QA_process 12:21:59 lbt: is there some important parts missing? 12:22:57 it's not 100% correct architecturally 12:23:30 but from a flow pov it's fine 12:23:40 4/5 is clearly not there yet 12:24:13 and I think we have 4 sent to vendors+maintainers 12:24:43 we previously discussed having 6/7 be a replication of 4/5 but delayed 12:25:10 so if a NO occured at 4/5 then the 6/7 subscribers would have lower test load 12:25:33 there's also a time limit on 4/5/6/7 for vendors ... 48h we mentioned 12:25:55 (you see why this uses a business process workflow tool :) ) 12:27:14 Vendor QA looks good 12:27:21 lbt: yes, I didn't draw the delayed notifications 12:27:23 * timoph needs to go 12:27:29 timoph: bye 12:27:34 timoph: o/ 12:27:35 o/ 12:27:54 from the vendor QA picture the qa-reports is missing 12:28:03 if the vendor wants to use it 12:28:21 yes, there are other things we'll add in more detail - such as updates to bugzilla when the patch is accepted 12:28:45 nod 12:28:48 currently the patch is manually accepted after Mer QA step 6 12:28:57 and that's worth noting I think 12:29:43 6 is actually more "update gerrit with results and notify someone to accept/reject" 12:30:07 (tecnically the maintainer can accept a package BOSS rejects) 12:30:24 I will add those to the description 12:30:58 *nod* so the "do stuff on accept" is actually a discrete process 12:31:47 that process is mainly for package change, the release testing process is almost the same 12:31:48 oh, we did do "add notation to mentioned bugs" at a step concurrent with 2 12:32:45 I'm not sure there is a release test 12:33:28 I think the notification should be something like "in pre-release mode" 12:33:30 but hmm 12:33:31 no 12:33:40 you're right 12:33:53 when we do a pre-release we do a changelog 12:34:02 that's a delta on package versions 12:34:04 in the release testing, the qa process is triggered by someone 12:34:15 that could/should drive the execution planner 12:34:24 yes, giving the list of delta packages 12:34:59 jumping directly to point 4 12:35:01 so vendor QA would be triggered in the same way but told "this is a snap/pre//release delta" or 12:35:36 wouldn't 2/3 still be needed 12:35:51 we're not testing images, we're testing releases 12:36:09 the vendor has a product release process which is a little different 12:37:22 does the vendor need own OBS too, if the vendor has to compile the package for his/her device? 12:37:34 we assume they'll have one 12:38:12 it is possible vendors will recompile all of Mer "to be sure" 12:38:24 we'd ask that this testing is done on Mer rpms 12:38:26 should that be on the vendor's picture, before step 2 12:38:41 I think we need another process there 12:38:50 ah 12:39:06 I see what you mean 12:40:10 the vendor might not use the Mer rpms 12:40:13 or cannot 12:40:22 mmm 12:40:32 1.1 would be BOSS triggers vendor OBS to use new package 12:40:39 1.2 BOSS waits for rebuild 12:41:14 yeah - we'd need to have some idea of what they're doing in that respect 12:41:29 and we'd assess their report based on our knowledge of it 12:41:41 where is the src of the changed package located, in gerrit and mer obs? 12:42:08 it should go via MDS 12:42:23 honestly don't know 12:42:39 one job I have is to build a reference implementation of all this :D 12:42:52 how it is done at the moment? the BOSS takes the src and creates a new package in OBS? 12:43:25 something like that 12:43:41 we may use copypac from Mer CI -> vendor OBS 12:44:25 and then giving the correct repository url to image creator 12:44:31 yeah - that would mimic how ci does it 12:45:12 I'd say boss gets packages from Mer CI (not using MDS?) 12:47:11 that will be one risk point, we will be wiser when we have first versions of image builder and test execution planner 12:47:24 oh yes 12:48:45 I was planning to setup OTS and nemo VM 12:49:00 then we would have something to work with 12:49:17 *nod* 12:50:13 (BTW, for the future, I'm hoping to get OTS to break down into functional process units that can be workflowed by BOSS) 12:51:27 can you explain that a bit more? 12:52:25 OTS uses much the same tech as BOSS to execute various steps - but AFAIK it's hardcoded 12:53:02 so I wanted to loosen the various steps to allow more flexible combinations 12:53:19 and to permit vendors to perform additional steps 12:53:35 it would also cut down on our maintenance burden 12:54:11 nb ... I consider this at the stage of "architectural blueprint" ... the real world may have different ideas :) 12:54:26 *nod* 12:54:45 there are many things what is needed for OTS 12:55:27 anything to add to the mer qa process? 12:56:16 nope 12:56:38 ok, I will wrap up our discussion then 12:58:20 yep - it's been good to expand on it a little 12:59:20 #info First Mer QA process, http://wiki.merproject.org/wiki/Quality/Test_processes#Mer_QA_process 12:59:23 #info Mer QA: Notification system missing 12:59:28 #info Mer QA: having 6/7 be a replication of 4/5 but delayed 12:59:28 #info Mer QA: if a NO occured at 4/5 then the 6/7 subscribers would have lower test load 12:59:31 #info Mer QA: a time limit on 4/5/6/7 for vendors ... 48h we mentioned 12:59:56 #info Mer QA: Other things needed to add in more detail - such as updates to bugzilla when the patch is accepted 12:59:59 #info Mer QA: Currently the patch is manually accepted after Mer QA step 6 13:00:02 #info Mer QA: 6 is actually more "update gerrit with results and notify someone to accept/reject" 13:00:31 #info Mer/Vendor QA: how to deliver correct version of packages to vendor, how the vendor compiles the packages 13:00:46 #info Changes to OTS, break down into functional process units that can be workflowed by BOSS 13:01:27 did I miss something? 13:02:25 I will update the pictures, and continue working with the test selector 13:03:14 #action E-P Update the QA pictures 13:03:36 thanks lbt for participating 13:04:08 np 13:04:11 i hope i have some time to try to test 13:04:22 this looks useful for setting goals+tasks too 13:04:42 so i can also check if the wiki-documentation is good 13:05:04 iekku: that would be nice, usually it is lacking 13:05:05 who manages OTS ? 13:05:41 lbt: no one i guess, i was planning to move it to the test-execution-tools gitorious project 13:06:16 I have to check that together with l3i 13:06:31 if they are still using it or developing it 13:06:51 we should definitely find out if there is an upstream 13:07:01 if not we can pull it in to Mer 13:07:11 meego.gitorius was/is the upstream 13:07:30 I'm torn - I hope there is no upstream so we can make it part of the BOSS code and manage it easier 13:07:41 :) 13:07:48 I will find that out 13:08:14 #action E-P find out who is maintaining the OTS and is there any upstream project 13:08:30 ok, have a great evening everyone 13:08:38 #endmeeting