12:00:17 #startmeeting Mer QA meeting 19/07/2012 12:00:17 Meeting started Thu Jul 19 12:00:17 2012 UTC. The chair is E-P. Information about MeetBot at http://wiki.merproject.org/wiki/Meetings. 12:00:17 Useful Commands: #action #agreed #help #info #idea #link #topic. 12:00:24 #topic Current status 12:01:02 I have done mainly preparations for this meeting 12:01:15 and packaged the mwts-telepathy, which needs lots of work in package level 12:01:41 hello 12:01:46 hi 12:01:46 hello 12:02:03 hi 12:02:13 hi all o/ 12:02:15 #info prepared: sp-endurance, sp-memusage, sp-rtrace, sp-smaps, swaplogger for later submission to :Tools:Testing 12:02:15 i'm waiting a visitor, so i might need to leave meeting 12:02:54 #info a test package for telepathy can be found from https://build.pub.meego.com/project/packages?project=home%3Aesmietti%3AQA%3ATests%3AMW 12:03:24 cool 12:03:39 phaeron: you were missing from the last meeting, could you update your status with virtual image testing? 12:04:12 #info most details of vm based testing worked out and implemented using imager 12:04:52 I can deploy it somewhere if the resources are available 12:05:19 do you have any doc how to do that? 12:06:00 * lbt is back 12:06:06 it's basically the same as imager deployment http://autodoc.meego.com/mint/imager/ , but I still need to generate specific docs and deploy it there. 12:06:12 lbt: nice to have you back 12:06:28 phaeron: ok 12:06:36 #link http://autodoc.meego.com/mint/imager/ 12:07:41 ah 12:07:43 sry 12:07:48 o/ 12:08:04 we can discuss details later and see if anything needs to be changed about the implementation to make it useable for community 12:08:04 phaeron: yes, I can probably work out some deployment there 12:08:11 does it do nested kvm? 12:08:43 i think iron for qa tests may make sense, as well 12:08:52 iron + a kvm on top 12:08:54 like with obs worker 12:09:45 ah 12:10:08 well, possibly - but that requires a whole phost 12:10:24 yes 12:10:27 lbt: same answer as you get everytime you ask the question. nested kvm is option in host so from imager pov it doesn't change anything 12:10:39 lbt: and it can run in parallel with kvm obs workers 12:11:08 that's how I am running it currently 12:11:45 I expected a "yes" but I know there had been some minor issues on nested img - wanted to ensure they didn't bite here... so thanks for the clarification 12:12:17 lbt: issues like the host freezing ;) 12:12:25 that'd be one 12:12:55 hasn't happened with recent kvm and kernel (opensuse 12.1 3.1+ kernel 12:13:11 *nod* 12:13:25 what else has happened in QA space.. we got copyprj going somewhat sanely but hit a roadblock 12:13:26 so I deem nested as usable but not sure production trustworthy 12:13:37 Stskeeps: that's a todo for coming weeks 12:13:57 certainly sounds like we'd want to avoid running it on machines that host key VMs 12:13:58 there's a mer prerelease out that we're testing atm 12:14:08 sounds good though 12:14:12 which may be a fail :) 12:14:13 o/ 12:14:17 also I did a couple of qa-reports deployments locally and got it going 12:14:39 did some css and template changes to get my hands used to it 12:14:52 and uploaded testrunner-lite results to it 12:14:58 going to write a client for it 12:15:05 to use the rest api 12:15:07 phaeron: you could create a patch of those changes or create a own repo for mer qa-reports 12:15:12 timoph or E-P: do you remember the graphing stuff qa-reports has? 12:15:16 is there any documentation of that? 12:15:41 what do you mean by graphing? 12:15:50 Stskeeps: iirc it uses google charting api or something 12:15:52 well, at some point, we had power measurements sent to qa-reports 12:16:04 o/ 12:16:08 Stskeeps: they are working pretty well 12:16:16 E-P: upstream is completely dead ? was hoping to collaborate 12:16:23 dunno about docs though 12:16:43 phaeron: I think not completely, mainly changing the meego stuff away :) 12:17:11 I think there isn't any docs about that, it uses the test results format for them 12:17:16 Paimen: could you talk about your TDriver stuff? 12:17:20 yeah obviously first thing I did was remove the hideous header and reclaim lots of vertical screen real estate 12:17:28 Stskeeps: well sure 12:17:54 not had too much time to go trough it, but I went trough qt5 branch in git 12:18:28 it has old naming conventions for QTquick 2.0 and last update is from august 2012 12:18:39 lbt: what is the status of the ruby packaging request? 12:19:03 so it should be went trough and look what changes are needed to do to enable qt5 testing via tdriver 12:19:33 Sage_: we agreed that getting BOSS for CI was higher prio - I'm still working on that. SDK and ruby packaging is next on my list. 12:19:43 what else. well Mer SDK tdriver host components seems to work ok at least with Nomovok bastard 4.8 qt running on iconia 12:20:12 #info TDriver works on mer SDK 12:20:24 also sut_qt plugin needs minor modifications to enable qtquick 2.0 12:20:27 lbt: k 12:20:38 #info phaeron is working with QA-reports 12:21:01 #info TDriver's sut_qt plugin needs minor modifications to enable qtquick 2.0 12:21:01 E-P: how to enable TDriver http://pastie.org/4248704 12:21:27 #info how to enable TDriver http://pastie.org/4248704 12:21:32 Paimen: thanks a lot 12:21:57 and major modifications to agent_qt client side 12:22:03 :) 12:22:40 setup that I tested had aget_qt 1.4.3 with qml2 branch merged and some fixtures modified 12:22:50 #info Major modifications is needed to TDriver's agent_qt 12:23:24 ok 12:23:44 something else, anyone? 12:23:49 EP dor QtQuick 2.0 that is, not tested pure Qt5, for that I'm planning to install nemo VM and work with Mer SDK to see how it works 12:23:54 s/dor/for 12:24:09 nod 12:24:11 I may have something relevant 12:24:19 there was a task about deploying qa-reports for mer 12:24:22 is that still needed ? 12:24:29 yes 12:24:31 yunta: mm? 12:24:31 yep 12:24:33 I've been refactoring testrunner-ui 12:24:53 ok I can set it up if I get a vm 12:25:02 I'm cooking OTS replacement with nice features that works with boss workflow. That includes "flasher" 12:25:03 #info timoph continues refactoring testrunner-ui 12:25:15 yunta: excellent 12:25:26 yunta: nice 12:26:01 can you tell more about that? 12:26:12 yunta: yell if you'd like any review of that 12:26:15 * timoph is all ears 12:26:20 hm 12:26:37 not sure where to start :D 12:26:38 just a sec 12:27:09 in general, we needed test distribution system that would work with boss 12:27:29 that would allow updates of server and workers without stopping test process 12:27:51 yes, that is big minus in ots 12:28:05 that would avoid unnecessary work (reflashing, package reinstallation,etc) 12:28:51 that would allow live status updates 12:29:01 and ots api is very limited 12:29:16 last time I used it at least 12:29:27 we're building it with phaeron for jolla 12:29:28 any estimates when we'll see it on action? 12:29:32 reduces technology variation too 12:29:41 and we totally want to contribute it 12:29:50 current status is: 12:30:03 prototype exists, works for vm 12:30:04 yes and my other items, are also jolla contributions 12:30:32 thanks to Jolla for that 12:30:38 avoids reflashing 12:30:41 yunta: good, I am interested on that 12:30:54 there is some ui for queue observation 12:31:20 web ui also allows scheduling tests out of boss process 12:31:23 one relevant point to (re)discuss at some point would be rewriting testrunner-lite in python 12:31:29 yunta, phaeron: I'd like to discuss the high level services/components you see being needed at some point - maybe a discrete meeting 12:31:33 somewhat related 12:31:38 yep 12:31:45 so you can run some more tests on your just-built image whenever you want 12:32:03 timoph: yep 12:32:15 timoph: our test distributor can execute any kind of tests, testrunner being just one of the kinds 12:32:35 we definitely have to chat more about that some another time 12:32:41 yunta: so one passes the executor to it? 12:32:57 #info yunta and phaeron are working on OTS replacement 12:33:04 if "Executor" is not a magic word than yes 12:33:10 :D 12:33:12 we call it Driver 12:33:19 E-P: it's mostly yunta :) 12:33:27 nod :) 12:33:36 but let's move on in our qa agenda 12:33:55 anyway, cool stuff 12:34:00 thx 12:34:16 let's have the OTS replacement topic for next QA meeting 12:34:21 ok? 12:34:24 sure 12:34:27 great 12:34:38 #topic Bug metrics 12:34:45 #link https://wiki.merproject.org/wiki/Quality/Metrics 12:34:58 those metrics are from Mer core 12:35:33 doesn't even look so bad 12:36:49 Sage_ is leading the fixing chart at the moment :) 12:37:25 he did in meego, mostly, too 12:37:26 :P 12:37:56 I would suggest that we update the metrics after a release is out, is that ok? 12:38:05 makes sense 12:38:23 yep 12:39:03 and would be nice if the release email would contain number of packages changed and changed, and number of fixes done 12:39:21 changed=added 12:39:50 iamer has been working on something like that 12:39:58 albeit for nemo 12:40:11 E-P: :P 12:40:35 Stskeeps: ok 12:40:40 (see nemo release notes) 12:40:43 i think 12:40:54 oh yeah , meego's repo diff is a good tool for that 12:40:59 I am extending it a bit 12:41:21 I'm redoing the release format for mer (there's a bug for it) 12:41:43 good good 12:42:09 anything to add to the bug metrics? 12:42:21 heh, probably should set assignee for bugs that I have actually fixed :P 12:42:32 https://bugs.merproject.org/show_bug.cgi?id=382 12:42:43 "Changelog summary is hard to read and needs to be online" 12:43:04 yunta: we could do with your bugbot too :) 12:43:24 not-taken and need-triage have done really good work :P 12:43:31 iekku: yep :) 12:43:47 E-P: http://pastie.org/4283588 but this is a bit artificial as I do my own snapshotting 12:43:50 lbt: bugbot? 12:44:04 bug 382 12:44:12 :( no url..... 12:44:18 :) 12:44:36 ah, nm, I meant jope :) 12:44:37 bugbot was useful 12:44:50 phaeron: something like that, yes 12:45:02 anyway, let's move on 12:45:23 #topic QA ToDo 12:45:30 #link https://wiki.merproject.org/wiki/Quality#ToDo 12:46:09 I was planning to file bugs/tasks after we have reviewed the todo list 12:46:14 I guess that has to be thought over if we're getting a replacement for ots 12:46:28 yep 12:46:55 the 1st priority is to get testing ongoing 12:47:02 yep 12:47:17 what's missing for that? 12:47:29 (to happen) 12:47:39 tests 12:47:43 and test plans 12:47:45 :) 12:47:48 lol 12:47:50 :) 12:47:50 i think a quick smoke test thing, "does this boot" is a good start 12:47:54 * Stskeeps looks at what he's debugging atm 12:48:16 we do play a game called distribution jenga each time we modify the core. touch one piece and everything can fall apart 12:48:28 :) 12:48:34 so in summary, the tools are there but no tests 12:48:47 timoph: pretty much, and infra is also missing 12:48:48 * timoph is having a meego dejavu 12:48:51 phaeron: didn't you have something where you didn't need to add a HW adaptation to test an image? 12:48:59 the vm based testing can help with basic boot test yes 12:49:05 that'd be a good start, i think 12:49:26 where we can even manually point it to a mer release snapshot 12:49:46 https://build.pub.meego.com/package/show?package=kernel-adaptation-vm-x86&project=home%3Aiamer%3AMer%3ASDK 12:49:57 this kernel does stuff 12:49:57 did it need anything installed into the actual image? 12:50:02 could we start filing bugs about what tests needs to be written? 12:50:04 Stskeeps: ssh 12:50:08 ah, that's fixable 12:50:11 and i guess eat-device 12:50:15 timoph, tasks yes 12:50:25 timoph: it is the 2nd item in the todo, test coverage 12:50:39 so, for next QA meeting, we can manually run a simple smoke test on a mer release and see if it still boots? 12:50:40 Stskeeps: not even that 12:50:56 Stskeeps: since minimal mer doesn't have Xorg etc .. 12:51:10 Stskeeps: http://autodoc.meego.com/mint/imager/configuration/build_image-conf.html#mic-and-mer-and-lvm minimal mer is already used as the imager kvm for mic image creation 12:51:16 :nod: 12:51:22 has someone tried filesystem-tests from mcts? 12:51:31 Stskeeps: the kickstart used is there 12:51:32 timoph: briefly, i think it mostly worked 12:51:34 iirc those were in pretty good shape 12:51:48 timoph: they should work, but the packaging needs some work 12:52:02 Stskeeps: http://autodoc.meego.com/mint/imager/configuration/build_image-conf.html#f2 12:52:08 I can start to work with test coverage 12:52:20 E-P: is there a task for that? 12:52:21 and some simple test plans 12:52:30 (filed) 12:52:31 timoph: not yet 12:52:41 all test packages need some work :) 12:52:51 :D 12:53:00 * timoph just needs a place to start 12:53:29 phaeron: do you have time to work with the vm testing and the automation of that? 12:54:18 E-P: I can invest time once copyproject is in use 12:54:25 as that is higher prio 12:54:28 phaeron: ok 12:54:36 I can deploy it if the resources are available 12:54:42 also qa-reports 12:54:55 phaeron: can you provide how to make image and use kvm to boot it, perhaps? 12:55:02 so others can look at it and work parralelly 12:55:03 (sp) 12:55:04 if I deploy it people can start playing with it at least 12:55:26 Stskeeps: the links I provided are a good start 12:55:27 ie, take this .ks and here's how to boot it with kvm.. 12:55:29 ok 12:55:30 phaeron: what do you need? a vm? 12:55:54 qa-reports -> vm , imager webui -> vm (can coexist) 12:56:11 lbt: you can help phaeron with that? 12:56:21 imager build_image -> host with kvm + lvm , imager test_vm -> host with kvm + lvm 12:56:36 nested kvm is a gamble but doable ( have it running on laptop) 12:58:02 ok, i think we can perhaps start out with having people using the kvm + .ks stuff (links above) and try to create tests for it 12:59:04 yes, after we have the test coverage chart it is easy to see what is missing 12:59:22 kyyberi: you are working on the dashboard? 13:00:10 yep to the vm 13:00:41 and I'm helping kyyberi by validating deployment of his meeting module 13:00:58 we'll KISS to start with 13:01:04 good 13:01:45 is the priority in the todo list ok? 13:02:08 E-P: yes 13:03:10 this weekend will be about doing the background jobs (some) to update data 13:04:14 ok 13:04:15 #info 1. Test automation: phaeron is working on copyproject, and deploying qa-reports and imager 13:04:24 #info 1. Test automation: yunta is developing OTS replacement 13:04:32 #info 2. Test coverage: E-P will creat test coverage page and first test plans 13:04:41 #info 3. Mer Dashboard: kyyberi is working on Mer dashboard and lbt is helping in validation 13:05:29 I will change the todo a bit because of the OTS replacement, and I will create tasks for the items 13:05:49 #action E-P update the todo list and create tasks for the items 13:06:25 do we discuss test selection stuff now ? 13:06:35 we are having bug squashing day on next wed 13:06:46 hope to get bz cleaned up a bit 13:07:21 phaeron: we can do that later when we have something working and some tests 13:07:28 ok 13:07:29 (I have to go soon) 13:08:08 iekku: that is good :) 13:08:40 are we done for today? 13:08:54 yes 13:08:57 thank you for hosting 13:09:00 yea 13:09:17 no problem, thank you for coming 13:09:26 #endmeeting