Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate switching away from GitHub #5205

Closed
nzakas opened this issue Feb 10, 2016 · 170 comments
Closed

Investigate switching away from GitHub #5205

nzakas opened this issue Feb 10, 2016 · 170 comments
Labels
evaluating The team will evaluate this issue to decide whether it meets the criteria for inclusion infrastructure Relates to the tools used in the ESLint development process needs bikeshedding Minor details about this change need to be discussed

Comments

@nzakas
Copy link
Member

nzakas commented Feb 10, 2016

As ESLint has continued to grow, we've started to outgrow the GitHub ecosystem. Team members spend hours each day triaging issues, many of which have incomplete information or are otherwise unclear. As such, we spend a lot of time just asking for more information ("what do you mean?", "what version are you using?", etc.). This has become increasingly frustrating for everyone on the team and ultimately takes time away from being able to contribute code to the project.

Additionally, it's nearly impossible to keep up with what are the most important issues and whether or not people are following up on issues. In short, everything that Dear GitHub mentioned is severely hurting the team now. As ESLint's popularity continues to grow and there are more and more issues filed, this is quickly becoming a scalability problem.

The team has discussed investigating alternatives to GitHub to see if we can find a home that is better suited to open source projects with our level of scale. We strongly feel that the code and issue tracker need to live in the same location to make it easier to manage and give people one location to visit for all of their ESLint-related needs (so simply moving to a different issue tracker and keeping the code on GitHub is not an alternative).

Requirements

  • Must host the repo along with the related tools
  • Must be able to run automated tests on pull requests
  • Must allow contributions from anyone
  • Must have a way to setup issue templates prescribing what fields are required
  • Must have ways to organize issues other than labeling (milestones, priorities, etc.)
  • Must import existing GitHub issues and pull requests

Alternatives to Investigate

  1. GitLab
  2. Phabricator

Concerns

  1. Will people find us in another location?
  2. How high is the barrier to entry for new contributors if we're not on GitHub?
  3. What about the Gitter chatroom? What happens to that? Do we need to find another chat location?
  4. Others?
@nzakas nzakas added infrastructure Relates to the tools used in the ESLint development process needs bikeshedding Minor details about this change need to be discussed evaluating The team will evaluate this issue to decide whether it meets the criteria for inclusion labels Feb 10, 2016
@ilyavolodin
Copy link
Member

Just to add a few more concerns:

  • Integration with other tools. We are currently use Travis, AppVeyor (both fall under need to build after each issue), but we also integrate with Coveralls (not really a deal breaker, but nice to have).
  • Even if people do find us in another location, would they be willing to create an account on the new service just to contribute to one project, or create an issue?

Maybe it's worth checking with other people who maintain large projects on GitHub and signed "dear GitHub" letter to see what their plan is? Coordinated move to the same alternative service would have higher chance of success.

@ilyavolodin
Copy link
Member

One other (pretty major) thing that I forgot is the ability to easily host/deploy site with support for markdown, as well as everything connected to that (plugins for markdown, custom domain name support, etc). I guess we could always go the route of separate website hosting, but it would mean more involved releases, and might be harder to maintain.

@nzakas
Copy link
Member Author

nzakas commented Feb 10, 2016

We could also keep the website hosting on GitHub if that's a big concern.

I'm not opposed to doing separate hosting, either. My personal blog is a Jekyll site that I update automatically. It would be pretty easy to setup the same thing for eslint.org.

@artyomtrityak
Copy link

@nzakas you can just move issues to Phabricator as babel did, keeping all popular projects in one place is really cool.

@RReverser
Copy link

As for organizing issues - can't you use ZenHub or alternatives so that repo would stay on Github? (doesn't solve the issue template though, but might be a good start and it's free for OSS)

Personally, I find it much easier to track all popular open-source projects in one place (that's pretty much the only reason I moved from Bitbucket to Github couple of years ago) and ability to participate in any of them through one UI, cross-links between projects etc.

Projects that move elsewhere raise the barrier (again, just a personal feeling / opinion here) to something like what projects on Google Code felt before its recent discontinuation or SourceForge feels like. Note - I'm not talking about UI, as other platforms can surely have features which are more convenient for maintainers, but rather talking specifically about the existing ecosystem and low barrier for participating in new projects within it as for newcomers as well for regular OSS contributors.

@captbaritone
Copy link
Contributor

I would like to echo @RReverser's concern about keeping the barrier to participation low. Participation comes not just in the form of commits and filing issues, but also staying abreast of what those issues are, and how they are being handled. I have an app on my phone that allows me to follow GitHub repos. I see all the PRs, all the issues, and all the responses. I see new commits and tagged releases. I'm sure there are many other people who are passively involved in the project via this or similar systems (including the builtin GitHub email notifications). The barrier to entry can be as low as clicking the "Watch" button on the GitHub page.

In another ecosystem, I'm guessing my best bet would email email alerts, and I have no interest in flooding my inbox with that number of emails.

Is it possible that we could take some work of the maintainers' plates another way? Maybe we could have some lower-lever volunteers who can handle the more mundane tasks?

I would really hate to see this project leave GitHub.

@afahim
Copy link

afahim commented Feb 11, 2016

As a fairly nascent developer, I think one of the things that I / people with my experience level care about is establishing credibility / getting credit for open source contributions, and whether we like it or not, github has become the first place to look whenever someone is trying to judge someone's contributions. It shows contribution activity loud and clear on profile pages, and this type of attribution is really helpful / important for new contributors to prove their mettle in the open source world. I feel that not having this kind of attribution would not provide enough motivation / demoralize a lot of people who would want to contribute, and I think this kind of lack of motivation and enthusiasm can be to the detriment of the project in the long term.

Given those considetations, I would really hate to see this project leave Github.

@mysticatea
Copy link
Member

Honestly, I love GitHub. I hope to not leave from here.

@alberto
Copy link
Member

alberto commented Feb 11, 2016

👎 to moving away, unless there are very clear benefits.

What is making us move? Is it some or all of the points of the "Dear Github" open letter? Anything else? Can we solve it without moving away?

@ljharb
Copy link
Sponsor Contributor

ljharb commented Feb 11, 2016

Not being on Github is a huge barrier to entry. Babel's move to Phabricator was perhaps a clear win for the maintainers, but I believe it's a huge downside for everyone else.

While it's important to make sure maintainers are able to sustainably maintain a project, it would be a very sad thing if users had to bear a burden because maintainers wanted to lighten theirs.

@janmarek
Copy link

I know the issues on Github are terrible. But the benefits of the Github community are great. Please keep in mind there is a barrier for new users - everybody is familiar with Github which is not the case for Gitlab nor Phabricator.

@MathRobin
Copy link

What about bitbucket?

@danpetitt
Copy link

Bitbucket seems to have a lot of downtime, I would suggest following their Twitter account to check it's not an issue for you

@BYK
Copy link
Member

BYK commented Feb 11, 2016

As a long-time Phabricator user, it looks very compelling to me but the above-mentioned barrier to entry is quite high, especially for new comers. Right now GitHub even allows submitting a PR purely from the web interface which may help people get started with quick documentation fixes. This is not possible with Phabricator.

Idea: move everything to Phabricator, including bug reporting but keep a GitHub mirror and only allow pull requests. This may address most of the complaints with minimal contributor impact. Since Phabricator supports many third-party logins, including GitHub, that shouldn't be a big deal.

It does support CI integration but may require some extra work to make the actual integration work. I'm willing to help with this if we decide to chose Phabricator.

@BYK
Copy link
Member

BYK commented Feb 11, 2016

travis-ci/travis-ci#2143 is the ticket for TravisCI - Phabricator integration. Looks like it won't happen soon unless someone tries to do it themselves. travis-ci/travis-ci#2143 (comment) shows a viable alternative.

@knowbody
Copy link

I think following Babel example and moving to Phabricator might be a good solution. Perhaps @kittens could share how easy/difficult was to move to Phabricator.

@saibotsivad
Copy link

I'd like to reiterate @RReverser's comment about Zenhub. I'm not associated with it, and I don't use it for any serious organization yet, but I surveyed it for use in our company and it solves a lot of the problems brought up in "dear github".

Note that one thing that it doesn't "solve" is that, since it's used primarily as a browser extension, viewers of your repo who don't have it installed won't be able to use any of the sweet features. For example, a user creating a new issue won't be presented with any configurable fields (e.g. "what did you expect to happen/what happened").

Still, you should give it a look and see if it might fit your flow to reduce "paperwork" costs.

@nzakas
Copy link
Member Author

nzakas commented Feb 11, 2016

I hear everyone's concerns about moving off of GitHub, it's not something I want to do. There's a clear barrier to participation elsewhere.

However, what we are doing right now is unsustainable. It's not just that the GitHub experience is suboptimal, it's that it aggressively undermines development. There's just no way we can keep up with the pace of issues anymore, especially when most of them lack enough information to be useful. The tooling isn't good enough for the onslaught we receive.

Really, it comes down to this: something has to change. It seems like GitHub won't change, or even respond to these concerns, so we have to do something.

Another option is that ESLint users step up and help us manage issues. I made that request in #5083 and got zero responses. If you're one of the people who commented about not wanting us to leave GitHub, are you willing to help out to make dealing with GitHub easier on the team?

Regarding ZenHub - looks interesting, though it doesn't solve our worst problem, which is issues filed without enough information. Also, I'd be looking at $50/month for the whole team, and more if we add more people to the team. It's not really economically feasible without sponsorship.

@nzakas
Copy link
Member Author

nzakas commented Feb 11, 2016

Oops, looks like ZenHub is free for open source: https://www.zenhub.io/open-source

@knowbody
Copy link

Zenhub is free for OSS

Edit: You beat me to it @nzakas lol

@joshmanders
Copy link

Have you considered a bot that would triage issues for you? Have it do some checks on the issue and respond with pre-configured responses and apply labels based on that? (edit: nevermind I just noticed eslintbot)

I highly support the dear-github movement, but it's a catch 22. Leaving GitHub and staying with GitHub both bring their own pros and cons. Unfortunately, all the cons of leaving outweighs the pros in my eyes.

@ilyavolodin
Copy link
Member

@joshmanders We already have a bot that does basic request for additional information. Most of the people simply ignore it:-(

@MoOx
Copy link
Contributor

MoOx commented Feb 11, 2016

Maybe working on the bot is not a bad idea. You could even auto close an issue if nobody responded to the bot request?

@joshmanders
Copy link

@ilyavolodin Have the bot pass through again after a time period and close the issue with a message.

@BYK
Copy link
Member

BYK commented Feb 11, 2016

Working on the bot also sounds like a good idea. It would automatically close the issue if certain values cannot be matched in the original issue message such as ESLint version etc.

@joshmanders
Copy link

@MoOx I have plans of creating a bot that does a lot of triaging and that for issues and help maintainers, unfortunately I haven't gotten farther than registering an account for it and start planning, but my goal is to create a bot that does a lot of offloading and providing it to communities to either host themselves, or remotely hosted.

I just wish I had more time to dedicate to it, because I know a lot of people are feeling the pains and could utilize it heavily and most people have their own bots that do a few things and that's it.

@ilyavolodin
Copy link
Member

We could beef up our bot to be much smarter, true. But in all honesty, in that case, we are going to be spending time developing bot's code, not ESLint:-( The other problem is that we already get complains from people thinking that our requirements are too strict. Having their issues auto-closed will only make it worse. I know that's not a good argument, but never the less, that's probably what's going to happen. And it does not really help team's productivity to see tweets saying how horrible and unaccommodating ESLint's team is.

@agilob
Copy link

agilob commented Feb 12, 2016

@agilob If Github doesn't work for them, how Gitlab would? I'm using it every day at work and it's more or less a Github clone. The only benefit is it's self-hostable and free.

@Bounga Because it covers their requirements in 100%

Must have a way to setup issue templates prescribing what fields are required

Supported, configured in repo settings, for both issues and merge requests.

Must have ways to organize issues other than labeling (milestones, priorities, etc.)

All supported by GitLab CE (gitlab.com). Issues also have weights, votes and belong to milestones, have labels and users can vote on issues.

Must import existing GitHub issues and pull requests

Not sure about pull requests, but issues and wiki are supported.

@MoOx
Copy link
Contributor

MoOx commented Feb 12, 2016

Pull request on GitLab are called "merge request" (which is the appropriate name according to the action) :)

@sarunast already posted.

@rafaelrinaldi
Copy link

Some great points on this subject from the folks behind successful open source projects such as Rails and Elixir (@josevalim): "Tips for keeping your Open Source Software issues tracker tidy"

Might help providing some insights.

@christophermancini
Copy link

We struggle with this too @ Basho. We are using JIRA and GH, which is kind of a pain. We have product development, custom development, and project management in JIRA while GH gets most of the bug & community stuff. We have a service that links the two, but the problem is that we are bouncing around, getting double notifications for things and it can create a confusing user experience for users in GH when an issue is updated automatically from JIRA because our PM changed something.

What we want is something like Waffle.IO that has PM capabilities + the ability to create private tasks / projects that don't get pushed to GH + ability to have a project that spans multiple repos. Haven't really found something like it yet.

@brainwane
Copy link

First off, thank all of you for working on ESLint!

I've worked on a bunch of open source and closed source software projects, using GitHub, Trac, Phabricator, Bugzilla, Mantis, FogBugz, and other issue tracking systems. Yes, none of them are perfect. GitHub has less customizability than, for instance, Bugzilla. I wouldn't be surprised if you do decide GitHub doesn't work for the particular workflow you want. (Are you using free GitHub hosting or have you paid for extra features?)

Bots and automation can help triage bugs, but in my experience there is no complete substitute for thoughtful people using their brains to address each one. Once someone gets the hang of it they can move pretty fast, and yes, this absolutely is an opportunity for new contributors. The Wikimedia "how to triage document" is a useful model here -- if you can add stuff like that to the "working on issues" HOWTO it'll help new people start as bug triagers. Wikimedia trained an intern as a bug triager and the Outreachy internships are a good match for this kind of work. Someone who starts by triaging bugs is often able to help a lot with release management, prioritizing features, communication with important upstreams and downstreams, and similar work.

I see some folks earlier in this thread suggesting better bug reporting guidelines and suggesting helping users with templates for required information. That is pretty helpful. I see some other folks using wording that implies that people submitting incomplete bug reports are stupid, lazy, or undeserving of help. That, in my experience, is an approach that leads to a bunch of less optimal outcomes. Every bug reporter is, by definition, running into a problem they don't know how to solve themselves; it's unlikely you're running into them on the best day of their life. I've usually found that a little human kindness and "could you go into more detail on points x, y, and z so we can have a better chance at solving this problem?" leads to bug information that helps developers make much more robust software.

Having a good short set of required data fields is useful. But it pays to be careful in the user experience around that. In my experience, it comes across as hostile to go with the "you didn't add required data, so we're auto-closing this, re-open if you think your new report is good enough" approach. That approach makes users more likely to think of maintainers as aloof and unresponsive; then, when a bug comes up that you really wish you'd known about, they don't bother telling you. A good middle ground is to ask for more data, and wait a few days, and then leave a kind explanatory message when closing the issue.

Hope this helps.

@Kmaschta
Copy link

I'm with @Nejuf and @JaKXz to suggest you waffle.io

waffle

For using it on a daily basis, it's pretty ergonomic and allows you to benefit the hosting power of Github without pain (or, at least, less pain).
I haven't tested it with a larger project as yours, but it deserve to be tested IMO.

@nzakas
Copy link
Member Author

nzakas commented Feb 12, 2016

@Marak Thanks for that, these suggestions are very useful. We can definitely make it clearer what is expected and how to request help.

You seem to want to fortify yourself away from the users so you can be shielded from their unbridled contributions and communications.

This is incorrect. We want a system that allows us better control of the contributions and communications. A side effect is that we'd lose some of it, but as I've said previously in this thread, I believe what we lose would be mostly low-level noise.

@nzakas
Copy link
Member Author

nzakas commented Feb 12, 2016

One thing I want to point out to a few people who have said things along the lines of "projects much bigger than you seem to be doing okay on GitHub."

Projects much bigger than us tend to have full-time developers and/or much larger teams. Node.js has not only a large number of contributors, but paid developers and the support of a foundation. npm has a company backing it. React/ReactNative have paid developers. The list goes on.

ESLint is run by volunteers working nights and weekends (and, to be fair, we also sneak in during the day to do some stuff when we probably should be focusing on our full-time jobs). The number of people who consistently make contributions is fairly low (< 10) and the number of people who actively help triage issues is even lower.

As I've mentioned earlier, part of the solution could be for more people to contribute. I've publicly requested help for issue triaging and had zero responses. Running an open source project doesn't mean contributors magically appear to help with everything, there's a lot of begging and pleading. (ESLint is certainly not the first project to face this problem.)

So while others have been able to make things work on GitHub, different constraints and resource availability mean that it's not a one-to-one comparison.

@mkurz
Copy link

mkurz commented Feb 12, 2016

GitHub responded to the "Dear GitHub" open letter: dear-github/dear-github#115
New features are coming to issues soon! Maybe wait for if them and give GitHub a second chance before switching to some other tool?

@young-steveo
Copy link

@mkurz #5205 (comment)

@cweagans
Copy link

Saw this on HN today: http://gitmagic.io/

Might be helpful.

@aldanor
Copy link

aldanor commented Feb 12, 2016

if GitHub indeed rolls out new issue features in the coming weeks, maybe that would be sufficient?

@brodock
Copy link

brodock commented Feb 12, 2016

GitLab have "Page Hosting" support like GitHub Pages. There is a WIP Merge Request to have CNAME and SSL supported for next release: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/173.

You can read more about GitLab Pages here: https://about.gitlab.com/2015/12/22/gitlab-8-3-released/

@nzakas
Copy link
Member Author

nzakas commented Feb 12, 2016

@aldanor yes, it could be enough to keep us around. I want to see what improvements are made before committing to anything, though. In the meantime, we're still going to be exploring other options so we're better informed.

@ShaneCurcuru
Copy link

The real question: is this truly a technical issue, or is it a community issue? I.e. are there ways to grow the community by using your governance and docs to draw in new contributors who can do useful work, not just submit incomplete tickets or the like?

Just a thought from the peanut gallery. If you're having this big a discussion on the details of tools that you use for the work, perhaps it's time to re-examine with a fresh eye how the social governance and interaction really works.

(Yes, I know, it doesn't always help, sometimes it's really hard to get new users to become contributors - but that's the only real way open source communities can grow over the long haul.)

Good luck!

@Qix-
Copy link

Qix- commented Feb 13, 2016

/quote

  • Must host the repo along with the related tools
  • Must be able to run automated tests on pull requests
  • Must allow contributions from anyone
  • Must have a way to setup issue templates prescribing what fields are required
  • Must have ways to organize issues other than labeling (milestones, priorities, etc.)
  • Must import existing GitHub issues and pull requests

/endquote

Seriously? What is on this list that Github can't do? You literally described all of the core features of Github.

What problems are you facing with Github, specifically?

Maybe we should consider BitBucket.

No, you shouldn't. If you think Github has abandoned its product, then you should see Atlassian's complete disregard for its product(s).


This is going to come across as being a jerk. Take it for what it is: direct, and subjective.

ESLint's governance is stiff and rigid. You guys have insanely high bars for what are considered "good" contributions. Maybe this is a good thing, maybe not - either way, your backlog isn't due to any sort of technical shortcoming, but a lack of involvement of collaborators (or maybe the number of collaborators?).

Another issue you're facing is an issue that most large OSS projects face - writing a PR for a feature/bug takes way longer than opening an issue, and most people are not willing to take the time.

I personally speculate that this is because OSS development has been gamified a bit too much. But that's just an opinion.

Switching to another tool is just going to derail this project beyond any sort of meaningful use. You're going to lose a buttload of contributors, watchers, experts, veterans and users alike.

You're also going to force contributors to learn a new product. However symmetrical Github and XYZ product are, you're going to be forcefully introducing human error. Isn't that what you're trying to "fix" in this issue to begin with?


👎 for this. If you do it, you'll lose quite a few users, contributions will diminish, and another, better alternative to your project will emerge.

Until Github itself dies, this is the best place for it to be. Github doesn't care if you leave. You're not sending a message by taking ESLint off of Github.

To me, this is trying to find drama where there is none and completely disregarding the practicality of migrating a project off of an established website considered 'home' since its conception.

That is my opinion.

@jamiebuilds
Copy link

So GitHub reached out and talked to me after responding to the dear-github letter earlier today. It sounds like the letter did more good than I expected it to, and all I'm going to say is that I'm looking forward to the things that they'll be doing in the next few weeks or so.

@nzakas I would hold off on making a decision for now. And you might want to consider locking this issue because it's turned into an exercise on who can write the most exhaustively long comment stating their uninformed opinion as fact.

@joshmanders
Copy link

Awesome news, and great work @thejameskyle!

@Qix-
Copy link

Qix- commented Feb 13, 2016

@thejameskyle that's exciting. Agreed on all points.

@nMustaki
Copy link

Hi,
what about having the bot sending a nice email directly to the bug reporter, asking him to respond the bot with a provided template ? It would then be posted to Github.
It won't be a silver-bullet, but it may be coded quickly and hopefully reduce at least a bit the unactionnable issues.

@joshmanders
Copy link

GitHub has added issue templates silently. dear-github/dear-github#125

All you have to do is add a issue_template.md file to your repo and when a new issue is created it should prefill in.

@brianjking
Copy link

@joshmanders It doesn't work with mobile access (Chrome on my Nexus 6P) or ForkHub on Android though, but it's definitely a start.

@joshmanders
Copy link

@brianjking indeed, just checked on my iPhone 6, it don't work. But agreed. Great start.

@nzakas
Copy link
Member Author

nzakas commented Feb 17, 2016

@thejameskyle as I mentioned earlier, we are continuing to investigate alternatives. While I'm happy about issue templates, I don't see any reason to stop looking into what other alternatives are out there. I've been having some great conversations with the folks at GitLab about their plans, and I'll post more about that when I have a second.

However, I am locking this issue at this point, not because I see any offensive comments (in fact, I'm thankful for the mature conversation that has take place on this issue), but because I feel like we have all the data from the community that we need. We know that moving off of GitHub would upset some, and that's why we're hopeful we don't have to do that. The team will continue posting updates here as things progress.

I want to especially thank the folks who also run large open source projects for their suggestions and sharing their insights. It's been invaluable to us.

@eslint eslint locked and limited conversation to collaborators Feb 17, 2016
@nzakas
Copy link
Member Author

nzakas commented Feb 22, 2016

I just wanted to add some data I got from GitLab:

  1. They'd be willing to give us an Enterprise Edition license so we could run it ourselves
  2. They already have an easy way to import a project from GitHub. The only thing that doesn't currently work is importing milestones: https://gitlab.com/gitlab-org/gitlab-ce/issues/13421
  3. They're working on a unified way to look at issues across repos: https://gitlab.com/gitlab-org/gitlab-ce/issues/2425
  4. The performance issues they mentioned on gitlab.com are now mostly resolved.

@nzakas
Copy link
Member Author

nzakas commented Apr 21, 2016

I want to thank everyone once again for the good discussion here. Since the time we opened this issue, GitHub has made a lot of improvements to issues and we've been better able to handle the volume of new issues we get. As such, we've decided that staying on GitHub is the right choice.

@nzakas nzakas closed this as completed Apr 21, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
evaluating The team will evaluate this issue to decide whether it meets the criteria for inclusion infrastructure Relates to the tools used in the ESLint development process needs bikeshedding Minor details about this change need to be discussed
Projects
None yet
Development

No branches or pull requests