# Preparing the DM Update for Scientists Session **URL:** \ **Category:** Data Management\ **Tags:** lsst2015\ **Created:** [August 15, 2015, 2:02am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72 "2015-08-15T02:02:09Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [August 15, 2015, 2:02am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/1 "2015-08-15T02:02:10Z")
Hi everyone (but (for now) especially @nidever, @RHL, gpdf, @jsick and @ktl ). There’s a session I’m leading in Bremerton next week on Thursday, called “[DM Update for Scientists](https://project.lsst.org/meetings/lsst2015/agenda/thursday/data-management-update-scientists)”. There are several goals: - (Re)introduce what LSST DM goals and what data products we’re going to produce (including pointers to where they can find out more) - Introduce the (new) DM team members to the science collaboration community, especially those who will be interacting with them the most (e.g., @jsick, @nidever, Colin Slater, etc.) - Give an update about new developments since last year, and plans for the following few years - Leave some time for discussion, Q&A Given that, here’s what I roughly have in mind: - [10 min?] Introduce new people - (maybe also mention new organization?) - [20 min?] DM Overview + dev plan (MJ) - For planning, I’d re-use much of the planning talk KTL will give on Tuesday - [10 min]? DM software/algorithm highlights in new release (rhl? @jbosch?) - [10 min?] Data we plan to process (DavidN?) - [10 min?] SUI plans (gpdf? xiuxin?) - [10 min?] New communication tools (jsick?) Questions for all of you: this is our opportunity to tell the scientific community what we feel they should know, and also for them to ask us questions and/or provide feedback. **_Is there anything else we really want to tell them?_** Or is any of the topics above not yet ready to be discussed? PS: This is just a 0th draft (that I’ll be updating tomorrow). But comments welcome! ---
**Author:** ![jbosch](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jbosch/32/12_2.png) [@jbosch](https://www.rubin.community/u/jbosch)\ **Post date:** [August 15, 2015, 2:18am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/2 "2015-08-15T02:18:03Z")
I have some slides from the recent HSC meeting on algorithm updates, focusing on the new multi-band coadd processing scheme, which I think could easily be adapted for this. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [August 15, 2015, 3:18am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/3 "2015-08-15T03:18:30Z")
I’m not giving a planning talk on Tuesday. I’m giving a status talk with a slide or two about short-range and long-range plans. So don’t expect much for the second bullet… ---
**Author:** ![jsick](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jsick/32/672_2.png) [@jsick](https://www.rubin.community/u/jsick)\ **Post date:** [August 15, 2015, 4:02am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/4 "2015-08-15T04:02:51Z")
Sounds good to me @mjuric. I’m happy get some face time so that folks have a point of contact to raise usability issues, among other things. In the last minutes I’d be happy to get people on [community.discourse.org](http://community.discourse.org) (pending RFC notwithstanding). I’ve talked to Willman and I may also talk about this site at either [LSST Communication II Tuesday 11 am - 12:30 pm](https://project.lsst.org/meetings/lsst2015/agenda/tuesday/lsst-communication-2-developing-best-practices) or [Informal Communication Lunch discussion Wednesday lunch 12:30 - 1:30 pm](https://project.lsst.org/meetings/lsst2015/agenda/thursday/lunch-informal-discussion-lsst-communication-scientists). A bigger audience could really help boost adoption, though. ---
**Author:** ![timj](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/timj/32/10_2.png) [@timj](https://www.rubin.community/u/timj)\ **Post date:** [August 15, 2015, 4:55am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/5 "2015-08-15T04:55:06Z")
> [@mjuric](#): > > Data we plan to process Throwing @nidever in at the deep end! > [@mjuric](#): > > New communication tools Are you just referring to Discourse here or do you have a larger scope for this? > [@mjuric](#): > > SUI plans (gpdf? xiuxin?) Gregory doesn’t have an account here yet but @xiuqin does. ---
**Author:** ![frossie](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/frossie/32/9_2.png) [@frossie](https://www.rubin.community/u/frossie)\ **Post date:** [August 15, 2015, 5:00am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/6 "2015-08-15T05:00:43Z")
@mjuric We talked about this with Beth last week, we’re happy to help but can you liaise with her to make sure there’s no duplication/overlap/mixed-messaging. She’s spearheading scence communications issues for LSST it seems. ---
**Author:** ![xiuqin](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/xiuqin/32/126_2.png) [@xiuqin](https://www.rubin.community/u/xiuqin)\ **Post date:** [August 15, 2015, 5:21am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/7 "2015-08-15T05:21:45Z")
@mjuric I will be happy to support this. How much detail do you want? ---
**Author:** ![nidever](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/nidever/32/29_2.png) [@nidever](https://www.rubin.community/u/nidever)\ **Post date:** [August 15, 2015, 11:52pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/8 "2015-08-15T23:52:38Z")
I’m happy to talk about “data we plan to process” as long as we can meet earlier in the week to discuss the plan. 🙂 ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [August 17, 2015, 7:56am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/9 "2015-08-17T07:56:52Z")
- @jbosch : yes, that would be great. I’d stick to astronomically obvious functionality as much as possible (i.e., multi-band coadds == go, refactoring of module X == pass) - @ktl : OK, I can prepare a brief summary (I’ll go over it with you). - @jsick, @frossie : I think we can use this to summarize where the project is going (if a clear direction crystalizes from Beth’s meetings over the past three days), as it pertains to DM. To be honest, I was initially thinking to use this to solicit feedback and/or further testing by the community for things like Discourse. Any other plans SQuaRE may have that impact the community, this may be a good forum to ask about it (e.g., software distribution plans, etc.) - @timj, @nidever : No worries, we’ve got resuscitation crews on hand :). On a more serious note, yes, we should meet before to discuss benchmark dataset plans. I’ll set something up for Mon, Tue, or Wed (may have to be over lunch or dinner) - @xiuqin: _If we’re ready_, it would be good to give the community a flavor of what our data access website may look like or what kind of functionality it may support (and invite feedback). Note the **_“if we’re ready”_** qualifier! If it’s better to hold off on doing this for a few more months until the plans for “version 1” crystalize, we should do that. We can chat offline. PS: I’m worried the list of things we wanted to present is too ambitious for the time allotted (1.5 hrs) – we may need to cut something ☹ . ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [August 19, 2015, 9:47am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/10 "2015-08-19T09:47:37Z")
Hi all (esp. @jbosch, @nidever, @davidciardi, @jsick), Just to confirm to everyone we’re sticking with the structure laid out in the opening post of this topic. The times listed there do **not** include the time for discussion. Also, if there’s any time left in the end, we’ll use it for an open Q&A session. PS: @davidciardi will try to deliver the talk via videocon; if that doesn’t work out, @xiuqin, could you do it (or nominate someone someone from the SUI team to take over)? PPS: If you want to dry-run (or have me look at) your slides, happy to do that tomorrow evening. Thanks!! ---
**Author:** ![xiuqin](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/xiuqin/32/126_2.png) [@xiuqin](https://www.rubin.community/u/xiuqin)\ **Post date:** [August 19, 2015, 3:14pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/11 "2015-08-19T15:14:42Z")
OK, I will do it if there is any issue with David’s remote delivery. @mjuric @davidciardi ---
**Author:** ![jsick](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jsick/32/672_2.png) [@jsick](https://www.rubin.community/u/jsick)\ **Post date:** [August 19, 2015, 3:21pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/12 "2015-08-19T15:21:08Z")
@mjuric Is a talk in the same spirit as the one I gave in ‘Communications I’ ok? Essentially: 1. introduce my role, 2. demo Discourse, 3. show slides inviting astronomers to join us and to think about adopting it for their own purposes. ---
**Author:** ![ctslater](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ctslater/32/28_2.png) [@ctslater](https://www.rubin.community/u/ctslater)\ **Post date:** [August 19, 2015, 3:57pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/13 "2015-08-19T15:57:49Z")
> [10 min]? DM software/algorithm highlights in new release One suggestion from a new-to-DM, science perspective is that it would be nice to have a sort of “checklist” of features that an astronomer might expect from a photometry package (not just a survey pipeline) that have or have not yet been implemented. E.g., - PSF photometry - Done - Coadd creation - Done - Moving object [something something something] - ?? - Deblending - ?? - Extended source photometry - ?? Overall I’m thinking if-this-was-sextractor, what would your feature list be right now, what is semi-complete, and what is the near term planned feature list. It’s definitely something that would help me understand the current capabilities of DM better. ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [August 19, 2015, 6:08pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/14 "2015-08-19T18:08:41Z")
@ctslater: “Done” is a somewhat problematic term. I prefer to think of things as getting to a minimally functional state, then a scientifically useful (state of the art) state, then a meets-the-SRD state. I’d like to eventually get to a color-coded chart that shows our progress on each component; I started working on that but got bogged down in tool issues. ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [August 20, 2015, 4:44am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/15 "2015-08-20T04:44:55Z")
Hi everyone, I have slides from @nidever and @davidciardi, I’m working to finalize my talk, and using the opportunity to ping @jbosch and @jsick to make sure they have theirs ready by ~9:30 am tomorrow :). ---
**Author:** ![nidever](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/nidever/32/29_2.png) [@nidever](https://www.rubin.community/u/nidever)\ **Post date:** [August 20, 2015, 4:48am UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/16 "2015-08-20T04:48:42Z")
I’d love to see that list (or color-coded chart) as well. ---
**Author:** ![jbosch](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jbosch/32/12_2.png) [@jbosch](https://www.rubin.community/u/jbosch)\ **Post date:** [August 20, 2015, 12:56pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/17 "2015-08-20T12:56:23Z")
Here are my slides; sorry I didn’t send them earlier. I’ll mention a number of changes we’re bringing over from the HSC side, then focus in on the new multi-band processing, which is both the biggest change (IMO) and the one for which I have the best slides. > **[pipeline-updates.pdf](https://www.dropbox.com/s/r0u8uc2la9uywst/pipeline-updates.pdf?dl=0)** ---
**Author:** ![jbosch](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/jbosch/32/12_2.png) [@jbosch](https://www.rubin.community/u/jbosch)\ **Post date:** [August 20, 2015, 1:00pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/18 "2015-08-20T13:00:07Z")
I think this would be very useful, and I’d be happy to put some effort into it myself - but it won’t happen by tomorrow, and probably not this week. Remind if I forget (unless @ktl or @mjuric want to take the lead on it). ---
**Author:** ![ktl](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/ktl/32/1373_2.png) [@ktl](https://www.rubin.community/u/ktl)\ **Post date:** [August 20, 2015, 2:48pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/19 "2015-08-20T14:48:17Z")
@timj has already been doing some work at [https://confluence.lsstcorp.org/display/DM/Data+Processing+End+to+End+Testing](https://confluence.lsstcorp.org/display/DM/Data+Processing+End+to+End+Testing) and the pages under it which he and I have used to start working on some diagrams. But we need to fix problems with them and then get to coloring. ---
**Author:** ![mjuric](https://sea2.discourse-cdn.com/flex002/user_avatar/www.rubin.community/mjuric/32/31_2.png) [@mjuric](https://www.rubin.community/u/mjuric)\ **Post date:** [August 20, 2015, 3:20pm UTC](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72/20 "2015-08-20T15:20:58Z")
Thanks everyone, Here are the presentations: > **[Bremerton-DM](https://www.dropbox.com/sh/ou9r9bcyvog6wah/AACg5mjB0Pv6syj79FEw-Acza?dl=0)** > > Shared with Dropbox You have ~40 minutes for a few final updates 😄 @davidciardi – We’re looking into setting up the videocon system in the ballroom; should have more for you in ~15-20 minutes. [Next page](https://www.rubin.community/t/preparing-the-dm-update-for-scientists-session/72.md?page=2)