2020-10-21 conda-forge core meeting

Zoom link What time is the meeting in my time zone last weeks meeting

Attendees

* Eric D
* Matt B
* Filipe
* Cheng Lee
* Isuru
* Marius
* Crystal
* Marcelo
* Paul I.
* Marcel B.
* Connor M.
* Bill Batey
* Stephanie
* John K.
* Lori B.
* Wolf
* CJ

Agenda

Standing items

intros for new folks on the call
- Bill - new to Anaconda. On the distribution team with Cheng and Connor.
(CJ) budget
- current approvals?
- Whenever updated numbers land, please screenshare and show the budget.
  - Link is in Keybase (numfocus_spreadsheets.txt)
(MRB) open votes
- Chris Burr for Core
- Won't use polys again because something totally went sideways. Maybe we'll try again?

From previous meeting(s)

(MRB) github releases and conda-forge-sparta
- code and data in the regro github org right now
  - if you want access, LMK
  - I am also happy to move the repos to the conda-forge org but they may slow rerendering since they use a fair bit of github actions
- how it works
  - all packages that are validated and copied to anaconda.org/conda-forge are also sent to a releases repo via a workflow dispatch event in github actions
  - the github actions CI job (regro/releases)
    - downloads the package
    - computes its repodata information
    - uploads the package and its repodata as a github release
    - pushes the repodata shard to another repo with all of the shards
  - another job on a continuous loop (regro/repodata aka the repoworker)
    - downloads the shards
    - figures out which ones are new
    - applies repodata patching to the shards (will also repatch everything if patches have changed)
    - computes the repodata and channel data
    - computes an index of links that point to where one can download the package
    - uploads all of this data as a github release
    - kicks the heroku server to download the new links
  - the herkou server (https://conda-forge-sparta.herokuapp.com/conda-forge-sparta/)
    - gets a request for a package or repodata file at the standard URL for an anaconda server
    - looks up where that package is in the links index
    - issues a 300 to redirect the request to the actual location
  - a final set of cron jobs runs once an hour to sync our repodata shards with anaconda.org (regro/repodata-shards)
    - this job catches any missed releases due to bad CI jobs or manual uploads
      - will update the shards repo and upload to github
    - ensures eventual consistency on ~1-2 hour time scales if things break
- overall time delay from the initial workflow dispatch to a package being available is 5-10 minutes
  - can go longer if repodata patches are being redone for everything
- how could this not depend on anaconda if we needed it
  - replace cf-staging and anaconda.org w/ quetz
  - limit traffic to those instances to keep the costs down
  - send the vast majority of download requests to our (possibly beefed up) heroku server and thus github
- I really don't want to maintain this for production, but we could if we had to.
- For now, we have a system to help index and archive our data in another spot.
(MRB / ED / SC) Roadmap / Funding? Discussion in gitter:
- (SC) What would CF do as a project if it received funding? Maybe having some kind of whitepapers on the conda-forge website would help. Would conda-forge do something very different if some corporation gave 10k, 100k, or 1M to the project via NumFOCUS?
  - 10k: does it buy enough hosting to do something interesting?
  - 100k - what does it bring.
  - A much larger amount: conda-forge may even be able to have funded people directly working on the project.
  - We can almost certainly find a lot useful stuff that would not hurt getting funding for without necessarily having to get consensus on a roadmap. (something i think would not be controversial, like oh we could use some funding to do some nice web development for better visualization of migrations, or stuff) there are probably a lor of things of that nature.
- If we make a list, Santa may just show up...
- see the bloomberg note above...
- wishlist and ideas: https://hackmd.io/0zGSUS71SbOdBsdLtDmGjg
  - (ED) Can we add initials for whomever submitted the idea? would be useful to track that if there are follow-on questions
  - (MRB) Start chatting on this for 15 minutes in the next core meeting
(MvN) tactical things to do about cython?
- CPython breaks its apis reguarly enough to force a recythoning of much of the python ecosystem.
- Should we patch our cython so that when the CONDA_BUILD env var is set, we always recythonize and ignore mtime for files?
(MvN) noarch: python v2
- As we increasingly drop compatibility with py27/35/36 should we automatically add python constraints to all noarch: python artifacts using repodata pacthing?
  - The idea would be to pick an epoch date (say 2020-10-01) and have the artifacts built after that date get a repodata patch adding a python>=3.6 constraint iff it only had a python constraint.
- Additionally we may as well upate the noarch adding bot action that adds noarch to strip all lines containing py27/34/35 selectors from the recipe.
  - This should allow us to move lots of feedstocks that currently depend on some things only for py27 compat to noarch: python cutting down on the growth rate of artifacts.
- to do
  - ( ) [ ] make a hint
  - ( ) [ ] make an announcement
  - ( ) [ ] make the hint a lint
(WV) multisheller github.com/wolfv/multisheller

Your new() agenda items

(FF) Outreachy, round 2
(FF) "Feature parity" between feedstock and staged-recipes. For example: CFEP-05 does not contemplate a dev release on staged recipes when there is no stable release to start a feedstock. https://github.com/conda-forge/cfep/blob/master/cfep-05.md
- No need for a conda-forge.yml, one can add a conda config file
- The broader issue still is not solved, we cannot do cos7
- staged recipes needs some work!

Pushed to next meeting

Active votes

Subteam updates

Bot

ARM

POWER

CUDA

Docs

staged-recipes

website

security+systems

CI infrastructure

Compiler upgrade

CFEP updates

Open PRs

cfep-04 X11 and CDT policy
- INACTIVE - Merge in with some inactive-esque status?
- Needs new champion. Thanks for your work on this pkgw! Has unaddressed comments from pkgw as from Jan 10, 2020
cfep-06 Staged-recipes review lifecycle
- INACTIVE - Merge in with some inactive-esque status?
- Lingering comment from @saraedum. @jakirkham, can you reply? Has unadressed comment from @saraedum from Jan 8, 2020
- (MRB) The stalebot has solved the worst of the issues here. I think we could defer this one permanently.
cfep-10 Feedstock statuses, unmaintained
- INACTIVE - Merge in with some inactive-esque status?
- Needs another review. Has unaddressed updates from pkgw as of Jan 11, 2020
cfep-12 Removing packages that violate the terms of the source package
- Stalled since May 26, 2020
- Active debate about moving to "broken" vs deleting from conda-forge channel
- Active vote, ends on 2020-03-11
- What were the results of the vote?
- Did we hear back from NumFOCUS?
cfep-17 Handling pin backports and dependency rebuilds
- Stalled debate about implementation details between Isuru, CJ and Matt
- UPDATE 2020-07-22: We in principle have agreement to render the extra pinnings needed directly in the feedstock on a temporary basis (i.e., until the migration has ended).
cfep-20

Discussion

Check in on previous action items

Copy previous action items from last meeting agenda.

This meeting

2020-10-21

Python 2.7 migration
- ( ) [ ] make a hint
- ( ) [ ] make an announcement
- ( ) [ ] make the hint a lint

Last meeting

2020-10-07

Make sure to add the NVBug info to the cudatoolkit package that conda-forge makes (if we make one)

2 meetings ago

2020-09-30

Move to Issue Tracker

2020-09-23

(MRB)
- do libgfortran name change
- add target platform to hashes
- do gfortran migration with bot
- bump pinnings

2020-09-16

Get a call set up with Jon Mease about the kaleido staged recipes PR
- Emailed on 2020-09-16
(FF) Open up a PR on the python feedstock for python 3.9 and see what fails

2020-09-09

(ED) Update governance docs with similar voting model as what got put into conda-tools (+3 with no -1 is a pass)
(SC) Write jinja template to turn institutional partners yaml into a website https://github.com/conda-forge/conda-forge.github.io/blob/2a2d3caaf7d74eb370ac40c679ba337a73d15c8a/src/inst_partners.yaml
(SC) Document what needs to be done to create an OVH account and get access

2020-08-26 Docker hub

(JK) Check in on Azure build workers to see if they have the docker hub limitation.
(JK) work with dockerhub to see if we can get OSS status
- Check in again at some point. We haven't heard back as of 2020-09-23
(MRB) start pushing images to quay (https://github.com/conda-forge/docker-images/pull/152)

OVH

Attendees​

Agenda​

Standing items​

From previous meeting(s)​

Your new() agenda items​

Pushed to next meeting​

Active votes​

Subteam updates​

Bot​

ARM​

POWER​

CUDA​

Docs​

staged-recipes​

website​

security+systems​

CI infrastructure​

Compiler upgrade​

CFEP updates​

Open PRs​

Discussion​

Check in on previous action items​

This meeting​

Last meeting​

2 meetings ago​

Move to Issue Tracker​

Attendees

Agenda

Standing items

From previous meeting(s)

Your new() agenda items

Pushed to next meeting

Active votes

Subteam updates

Bot

ARM

POWER

CUDA

Docs

staged-recipes

website

security+systems

CI infrastructure

Compiler upgrade

CFEP updates

Open PRs

Discussion

Check in on previous action items

This meeting

Last meeting

2 meetings ago

Move to Issue Tracker