Personal tools
  • We're Hiring!

You are here: Home Community Minutes Conference Calls 2009 2009-12-15 Tuesday Meeting

2009-12-15 Tuesday Meeting

Agenda

  • Post-ASCB activities
  • 4.2 roadmap
    • Getting HCS preview back up
    • Suggestion: permissions/ldap/ssl (across all clients), HCS (across al clients, effectively HCS_PREVIEW is released), Open GL, bf cleanup, some scripts
    • Simultaneous progress on bigs topics (see next item)
  • Beyond
    • NDIM, BigImages, Data duplication
  • FS talk (Colin)

Notes

  • For 4.2

    • Add "work-in-progress" / "preview" back to the downloads page
    • "sharing" (access to other people's data) obviously necessary
    • polishing rois, scripts, etc.
    • requested last time at Paris
      • Data duplication (should be in pipeline)
      • DataIn/Out
      • Analysis
    • Chris: Deployment issues
    • Data duplication before NDIM & BigImage?
      • Probably
      • Jason: folks act like infinite file system
        • only work with small percentage of data
        • data generated as your doing an experiment
        • need to modify requirement that everything is inside OMERO
        • 90% of metadata is otherwise crap
        • impedes performance
      • Jason: N screens have to go in quick
        • have to get people past idea that a single application for all their data
        • should only be high quality which goes in
      • managing expectations
        • "give me everything i want with complete flexibility and super fast"
        • penalties for certain usages
      • fs-solutions are untenable because sysadmin is bottleneck
        • user needs flexibility to point at any filesystem
    • End of April: 3 months of coding & 1 month of testing
      • permissions, ldap, ssl
      • bioformats cleanup
        • caching initialization
        • reader specific optimizations
        • common classes
      • preview mode on the scripts (few new ones)
      • what to show for analysis? needs to be articulated
      • (preview stuff every few months even without a release)
  • FS (Colin)

    • obvious issues
    • solution #1: deduplication using hashes on blockes
      • uses sun processors for hashing
      • OS support is dodgy
      • doesn't solve the parsed data duplication
    • solution #2: only import metadata
      • file renaming, etc. deletion?!
      • modifications allowed? does what their software allows them to do
      • some folks at ASCB accepted the fs mount being read-only
    • solution #3: having fs servers talk to one another
      • fs smarter
      • needs db access
    • solution #4: hashing
      • needs significant testing
      • is the cost of finding where to calc the hash more expensive than the hash
      • first plane or metadata? on metadata modification?
        • only changing the metadata in the db
      • how often are files moved/changed? (lively fileystem) more than import overhead?
        • spring cleaning; have to handle this well
        • also: archiving
      • multi-file hashes!
    • solution #5: joining the paths/shas/db back up in the client
      • trying automatically is difficult
      • have to make it very clear.
    • archiving
      • clients will have to be aware
      • there will have to be exceptions (and a service?)
    • initial possibility: server-side filesystem viewer with manual "import"
      • user clicks on file and "registers" it with server (file is hashed, etc.)
      • no notifications
    • proxy on the client side for mounting a directory?
      • all IO is redirected locally
    • further discussion tomorrow (Wed, 1000 UK)
Document Actions