Remember: Agenda must be complete (with estimated times) on the day before the meeting. Any additions after that must go at the bottom (AOCB)

Agenda - 2:30pm Start

Dundee: Kenny, Chris, Jean-Marie, Blazej, Andrew, June, Emil, Gus, Petr, Helen, Ola, Simon, Roger, Will, Mark Remote: Colin, Sebastien, Sebastien, Josh, Josh, Mark H, Douglas, Melissa, Carlos, Stick

1. Accepting minutes from last meeting

   • Accepted

2. Welcome June Matthew (new OME Admin in Dundee) and Joshua Ballanco (Glencoe)

   • June: worked at university for 10 years
   • Joshua: lives in Turkey. PhD in evolutionary biology, then dev..

3. Testing update/project status

   1. QA tool see report. We are going to use Squash
   2. FS non FS testing see Plan
   3. Squash not ready for testing just yet - needs time to get set up. Gradual migration.
   4. Take 4.4. DB and upgrade to 5.0 each day.
   5. Start small - increase data amount to cover as much as possible.

4. Glencoe update - 14:37 UK

   • Nothing beyond last week

5. Consortium update - 14:37 UK

   • Nada

6. Upcoming meetings/conferences/outreach visits

   • GRE symposium - November 22nd
     • Present various OMERO tools on poster. U-track & FLIMfit
   • 2 weeks, biobank presentation on 19th Tuesday meeting.

7. JCB DataViewer presentation - Chris Allan (15 mins plus discussion) - 14:41 UK

   • manuscript: 201303145
   • challenges of working on top of open source
   • Thanks to Glencoe team - big now. Most tasks distributed
   • OMERO 4.4.x, Custom Importer, BF fixes back to open source
   • Blitz gateway API is critical
   • JCB application is full standalone plugin
   • webpublic infrastructure used for public
   • Screens use OMERO.tables and search.
   • No Insight / webclient usage.
   • JCB data maps onto standard OMERO data types & annotations
   • Lots of OMERO tech used to generate SPW pages, charts etc
   • Numbers for latest screen are big - 1.3 TB. Many metadata objects. E.g. 482416 Channels
   • Process: 35 hours to import --no_thumbnail is critical
     • populate_metadata - 20 mins
     • Annotate Screen
     • Adjust custom index bridge - reindex 3.5 hours
     • Set image metadata. Channel names, rendering settings etc. 150ms per image = 11 hours!
   • Lessons:
     • All components are deployed separately.
     • Limited number of workers: 8
     • Don’t thumbnail on import
     • Offline indexing (complete DB replica)
     • Large annotation counts suck. JOIN across 100,000s of objects
     • Avoid omero.gateway for performance critical sections - Use custom queries, IQuery.projection()
     • Template optimisation. “with” keyword vital in some areas. DRY!
     • More advanced Django worker and nginx setups. zlib compression, more workers & longer lived. Gunicorn
     • Separate OMERO.web deploy highly beneficial - 2-3 times a week.
     • webgateway_cache needs a lot of work (in progress)
     • CLI critical for large datasets, hql, delete, chgrp etc
     • Downstream of dev_4_4. Merge points need to be clearly defined. Parallel builds
     • OMERO.web plugin config - needs discussion
     • Need to consume JARs via dependency resolution
   • Points to discuss
     • HCS tools used cross consortium - large datasets - testing
     • Tension between “gateways” and performance
     • Large datasets can break standard clients
     • Future of rendering settings and rendering engine API
     • Future of S-P-W-WS-I model. Need to load lots of data to get plate dimensions etc
     • Backporting - web plugins, worker config, caching, lib/repository and target/ layout, Ivy modifications

8. Questions

   • Ola - list of performance changes backported? Chris - need to review
   • JM - Re-use Channels across plate? Chris - could do that now (data reduction)
   • Still have LogicalChannel? (removed from OME model a while back)
   • Andrew - How to fix rendering settings? Chris - fewer rendering settings - 1 per image / plate. Andrew - rdef needs to be in OME model, for export etc. Chris - JCB don’t want 1 rdef per user. Will - can’t show custom rdef on public data
   • Will - lots of effort to provide data, but users can’t really use the data (just browsing thumbs etc) Chris - Graph is most useful - browsing plates/thumbs is more of a gimmick
   • Roger - Load plate of thumbnails as one image? Chris - Looked at this but image is big, slow (don’t see thumbs loading) Client-side plotting etc browser support improving - but still 25% IE8!

9. Any other business (<5 mins)

   • Build email discussion (Chris/Josh)
     • omero-team currently set for a number of jobs. Getting too many e-mails on failing builds.
     • Need more flexible e-mail lists for builds.
     • Propose new alias "ci-status". Who would join?

10. Youtube for demo videos (of not released functionality) - Discuss next time!