2009-09-04 OME Group Conference Call
Attending: Andrew, Colin, Chris, Brian, Curtis, Melissa, Donald, Ola, Jean-Marie, Will, Josh
Agenda
update on XSLT fun (Group A)
QA
- status
- todos
Permissions
- Todos
- Priorities
Analysis Preview
- Rois : Ellipses!, Masks?
- Measurements : service, parsing
- Formats : directory import, dropbox
Data Model
- Filter, detector proposal
4.1 schedule
- Graffle : usable? offmark?
- iteration 1 : remote dropbox, rois, permissions, validator (on track? date?)
- iteration 2 : codegen, xslt, qa, ... what else? starts?
-
Missing from plan
- DB Upgrade : new formats? freeze?
forums and emails
Notes
xslt (Andrew)
- transforms in place that work for given data
- 2003, 2007, 2008-02 --> 2009-09
- don't have tests written
-
broken: roi in source file is rubbish, get gibberish out
- still written under limitations
- loop options for z&t --> stuck in continuous loop
- need tests for that
-
another week for all of them?
- all transforms done.
- write tests, fix roi
- need help? probably. (input/output files validation)
- reusable chunks: need to refactor.
QA (Ola)
-
metadata-validation collector is done
- checks all required/optional(linked) fields
- doesn't do this via OME-XML but via models.
-
next phase is to check the data
- not checking the fields currently
- currently checks timestamps and structure (proper links)
-
Chris: don't worry about displaying the output?
- Ola: perhaps a simple display to know we're collecting properly
- parsing XML file with 2000 record is ok. how fast with 10K? more
- need testing with Chris, but generally works.
- need more time for displaying the collected data
-
currently limited by a memory leak
- can only import about 1000 files
- @Brian will add a ticket
-
cF. previous validator was just for XML/TIFF
- two different things
- review when OME-XML generation is always valid
-
Curtis: providing these tools to do all these checks
- warnings for LSID, etc.
- Adding xml generation to metadata-validator calls!
-
NB: should be straight-forward to take an unordered XML
- and produce an ordered XML
- Will: or removing the sequences?
- Curtis: Xslt would be fantastic (short to mid-term)
- Moving comments/bugs to new system?
- data collection: only we will be able to see data.
-
Will: moving all the existing feedback?
- Not migrating everything. Starting fresh.
- Everything turned into tickets? Yes.
- Linking between trac and qa. (have a separate trac for testing)
-
metadata-validation collector is done
permissions (Brian)
- private space, and a single or multiple public space?
- fine for vast majority of people
- private space gets rid of PI concept. perhaps via 600
- Donald: no private space?
- Brian: can you add more than one boss?
- Chris: how does this work with the setting a group to public from private?
-
quickest option:
- uesr only in one group
- turn group private or public
- server ensures graph-consistency, both group_id and permissions
- PI or admin can move group to public
- UI: need to know group prespective (don't show data)
-
ADMIN UI: need to allow upgrade to group visible. can't downgrade
- offering upgrade button
- sending email to all the users??
- next stage: either private space or multiple groups.
-
really a testing issue: moving the component/client tests.
- get them green
- then modify them as expected
analysis preview
-
import (Chris)
- getUsedFiles is populating, only on images themselves.
- big hash table is now associated. (great!)
-
Questions:
- two hashtables: global metadata & series metadata * combined together, since many don't have series * a bit of data duplication. shouldn't do any harm.
- every image will now have an annotation (yellow flag) * do companion files count as annotations? * already ruins the use. * ignoring file annotations? No. That's not good.
- adding relevant file formats to db
-
rois
- companion files have roi data (geometetries)
- overlay files: masks
- ellipse rois now displaying in the client
- measurement tool will be read only.
- extremely slow to get from the server.
-
mias
- 2 sets of data. tab separated rois (assumed circles)
- tif file overlays. rois drawn in red/green/magenta/...
- no correlation between the two
- adjusting pixel infrastructure with bit depth * take each color as a mask * show in rendering engine as channel * ..or.. insight as an overlay
- files have a color map; last five elements in the color map
- parsing it out is simple.
- overlapping? yes.
- talk @ 1400-1500 (MAX!) BST for rois
-
dropbox for hcs
- subclassing file monitor, waiting for no activity or a specific file
- then messages the monitor system
- chris: start work on directory-based import next week
-
Curtis: metadata as attachments. Why not as SAs?
- Chris: we are using SAs. Just a file-based SA. (i.e. performance)
- XML file later, if we want.
- Jean-Marie: editor! :)
- Curtis: easy to have editor ready k-v pairs. ini headers? Yup.
-
Curtis: roi coordinates
- massaging based on other formats?
- same case in 2D as 3D?
- define shapes differently. define in schema.
- need to continue conversation later. - imaris does things in a particular way - need to review what others do. - making the pixels selected as closely as possible. - two different definitions for each geometry? - massaging/transforms? - well-documented in the schema! (current and future)
-
import (Chris)
data model
-
filter proposal
- few people read it. few suggestions from curtis (remove attributes)
- shorter proposal for filter
-
detector http://www.ome-xml.org/ticket/65
- is more of the start of a discussion
- did anyone look at it?
-
Andrew: why do need the light path object?
- contents could be in channel itself.
-
filterset? very specific thing.
- is it the term for a type of filter? don't think so.
- curtis: still a valuable concept. need a place to store that
-
jean-marie: keep filterset and add phase (collection of filter)
- each linked to channel?
- everything is duplicated
-
Andrew: putting it on instrument and reusing?
- everything else is
- Will: but more like detector settings.
- filterRef is becoming a filterSettings object
-
planning other meeting
- proper XML examples
- curtis attend or review after the fact? will try to be there.
- 1400-1500 (MAX!) on Tuesday BST
-
filter proposal
4.1 plan updates for week two
-
opinions
- graffle nice. update it? yes
- xslt tests, ordering
- metadata-validator xml-generation tests
- qa: finishing xml validator, update registry, changing comments over? reviewing UI next week (wed)
- perms: josh/brian moving tests from components/client
- evaluating file annotation filtering in counts.
- file formats in db
- rois: speed, overlays, definition review
- dropbox for mias (in progress)
- directory import (next week)
- db upgrades
- new measurement ("Tables") service for beginning of week
- confocal metadata completion
-
opinions