2009-12-15 Tuesday Meeting
Agenda
- Post-ASCB activities
-
4.2 roadmap
- Getting HCS preview back up
- Suggestion: permissions/ldap/ssl (across all clients), HCS (across al clients, effectively HCS_PREVIEW is released), Open GL, bf cleanup, some scripts
- Simultaneous progress on bigs topics (see next item)
-
Beyond
- NDIM, BigImages, Data duplication
- FS talk (Colin)
Notes
For 4.2
- Add "work-in-progress" / "preview" back to the downloads page
- "sharing" (access to other people's data) obviously necessary
- polishing rois, scripts, etc.
-
requested last time at Paris
- Data duplication (should be in pipeline)
- DataIn/Out
- Analysis
- Chris: Deployment issues
-
Data duplication before NDIM & BigImage?
- Probably
-
Jason: folks act like infinite file system
- only work with small percentage of data
- data generated as your doing an experiment
- need to modify requirement that everything is inside OMERO
- 90% of metadata is otherwise crap
- impedes performance
-
Jason: N screens have to go in quick
- have to get people past idea that a single application for all their data
- should only be high quality which goes in
-
managing expectations
- "give me everything i want with complete flexibility and super fast"
- penalties for certain usages
-
fs-solutions are untenable because sysadmin is bottleneck
- user needs flexibility to point at any filesystem
-
End of April: 3 months of coding & 1 month of testing
- permissions, ldap, ssl
-
bioformats cleanup
- caching initialization
- reader specific optimizations
- common classes
- preview mode on the scripts (few new ones)
- what to show for analysis? needs to be articulated
- (preview stuff every few months even without a release)
FS (Colin)
- obvious issues
-
solution #1: deduplication using hashes on blockes
- uses sun processors for hashing
- OS support is dodgy
- doesn't solve the parsed data duplication
-
solution #2: only import metadata
- file renaming, etc. deletion?!
- modifications allowed? does what their software allows them to do
- some folks at ASCB accepted the fs mount being read-only
-
solution #3: having fs servers talk to one another
- fs smarter
- needs db access
-
solution #4: hashing
- needs significant testing
- is the cost of finding where to calc the hash more expensive than the hash
-
first plane or metadata? on metadata modification?
- only changing the metadata in the db
-
how often are files moved/changed? (lively fileystem) more than import overhead?
- spring cleaning; have to handle this well
- also: archiving
- multi-file hashes!
-
solution #5: joining the paths/shas/db back up in the client
- trying automatically is difficult
- have to make it very clear.
-
archiving
- clients will have to be aware
- there will have to be exceptions (and a service?)
-
initial possibility: server-side filesystem viewer with manual "import"
- user clicks on file and "registers" it with server (file is hashed, etc.)
- no notifications
-
proxy on the client side for mounting a directory?
- all IO is redirected locally
- further discussion tomorrow (Wed, 1000 UK)