Allow resurrecting tombstoned package names?

[kaste]

The crawler currently tombstones removed packages, which means their names are marked and cannot be used again. (This is unrelated to the sec issue with GitHub repos.)

Usually there are enough names available so that shouldn't be an issue but some notes about the flow here:

# We generally only crawl registered packages
for entry in packages
# t.i. we don't crawl un-listed packages, 
# but when we resurrect them, we would
# but we never revoke the "removed" key from "workspace.json" set in
maintenance()
# so these packages will not arrive in channel.json
#
# (a) either do not crawl removed packages at all
# or (b) unset "removed" as soon as we hit "crawl_package()" 

Sec note: whatever we do, we can only resurrect/reuse names registered in our officially maintained channel. We actually tombstone because third party repositories could steal used names.

[braver]

If someone wants to publish a package with name X, searches for that name and finds out there is a dead package with name X, we currently tell them it’s ok to take over that existing entry, if their new package at least also has a similar feature set. Just like if it weren’t dead completely but just unmaintained. As you say, it is only allowed if we review and give permission, and thus only on the public channel.

[kaste]

@braver That probably never happened, I don't think wbond had a notion of tombstoned packages, and some sec issues around that. Here I mean: using a name that is not in the registry anymore, a removed package, for which we should still have a site, see #64. Maybe these packages should remain in the registry and we would just add a "removed" key to them, t.i. we should not remove them at all. But at least the workspace.json is already only ever growing and never forgets a used name.

[braver]

using a name that is not in the registry anymore, a removed package

Packages never get removed from the old site even after removal from the registry. Or not automatically at least. Example: https://packagecontrol.io/packages/TortoiseGIT, sublimehq/package_control_channel#9114. Whether in practice it ever happened… not sure, I think I’ve seen it at least once.

[kaste]

Ok, sorry for the noise. Code wise, that is effectively just an if statement that you need to get right, (so don't do this drunken). I can add this, if we want to, and remove the "discussion" label.

[braver]

so don't do this drunken

I'll try to keep that in mind 😂

I notice some removed packages have at some point also been removed from the site.. not sure how that happened, perhaps Will did some housekeeping at some point.

So, yeah, in short, a tombstoned package name should not be up for the taking, because the name used to mean something and it can't suddenly mean something else, but it should be possible for us to allow it to be taken on the default channel after review and approval.

[braver]

@kaste where do we stand now with regards to this and #64? If something is removed, we currently keep them in the workspace at least, and displayed on the site. So we know it's there and if someone want to take the name we can potentially comply assuming we can trace back what the original package was about and if that makes sense. However, I do see in many cases the original repo was removed as well, so we're completely reliant on GitHub keeping the PR that added the package, and us being able to make sense of that. ... It's very much an edge case.
I would personally perhaps go with this: if a package gets tombstoned it cannot be resurrected. To replace a package with a newer fork, you try to do so before it dies. If the situation ever comes up that someone wants a tombstoned name and makes a very good case for it, we can always investigate at that point what to do about it.

[kaste]

Right now 404's (e.g. deleted repositories on Github) are a final state. And removed from the registry too. For the registry removals, we need a notion of safe channels, this is the main channel, and the LSP and SublimeLinter channel. All others are not curated and we cannot allow that others claim a used name: User foo removes package bar, user zoo registers bar on a private channel. This is not allowed anymore. (It was before!) I have this restricted because I need to write test around the desired behavior. I have a test system running. We just need to write the test that clearly illustrate the state changes in registry and workspace and what exact behavior we want. The time consuming thing is the decision-making. I have a todo list on my computer. I don't post it here as it would immediately show the attack vectors. During my vacation something happened I thought I would've catched and disallowed. I still need to figure out if I have a bug, or if it was just in an early state of thecrawl. We really need this notion of safe and un-safe channels. I think we need to state that names are not only unique but the pair (name, channel_name) must be protected. The channel name MUST not change or changes to a safe channel. I think that is the right model. And that is not in the code right now.

[braver]

I think we need to state that names are not only unique but the pair (name, channel_name) must be protected. The channel name MUST not change or changes to a safe channel.

Excellent, 100% agree on that. Names mean something and cannot be taken, names can move to or between the trusted channels, but not out of them. Makes sense.