Setup documentation should be embedded during source and destination configuration

[avaidyanatha]

Main Issue

We run into a lot of issues that can be simply explained by having documentation next to source and destination setup that clearly explains what each field is and how it is relevant to the connection. This has been requested and will save us a lot of time answering questions that can live within our UI.

Fivetran's Approach

This is how Fivetran does it. We should make it very simple and just do it in order of connector inputs, as the order of connector inputs should follow chronologically in setup.

Where will it live?

Each setup guide will live in its relevant connector catalog as a setup-guide.md file. We will split out the connector documentation into three files:

• overview.md
• setup-guide.md
• changelog.md

As it will be nested, this won't create noise on the documentation site unless a user is opening up many connector pages.

• Split out connector documentation into Overview, Getting Started, and Changelog.
• Evaluate how we can get all the docs onto the server docker image
• Implement a markdown reader
• OAuth?

[jrhizor]

It'd be great if external connectors could package this or link to the markdown from their specs.

[cgardens]

@johnlafleur do you have an opinion here? I think we had considered this at some point but decided it didn't look great. I'm definitely opening to reconsidering, but wanted to make sure you got eyes on it.

[cgardens]

@avaidyanatha i'm not fully understanding the "Where will it live?" part. Let me try to state in terms of what I think you're going for in user experience.

• If opened in docs.airbyte.io all 3 sections should be displayed overview, setup guide, changelog
• If opened in the app next to the configuration form, just setup guide should be displayed (or all 3)?

[johnlafleur]

We discussed about it last week. The reason for this change is that developers will often not click on documentation link, they will try to do without, and actually they might also miss the link after a while trying it.
There is a 35% drop between the moment they select the source connector and getting it validated to be redirected on the destination page. So the goal is to reduce this to a 20-25% drop instead.

[johnlafleur]

The issue I see with this approach is that our current setup guides in the documentation is not very helpful, and refer other links if you want to know more. If we want to go this way, I feel we need to make the setup guides more explicit (possibly with screenshots). Wdyt @avaidyanatha ?

[avaidyanatha]

@avaidyanatha i'm not fully understanding the "Where will it live?" part. Let me try to state in terms of what I think you're going for in user experience.

• If opened in docs.airbyte.io all 3 sections should be displayed overview, setup guide, changelog
• If opened in the app next to the configuration form, just setup guide should be displayed (or all 3)?

@cgardens - Yep you nailed it. If opened in docs.airbyte.io, all 3 sections will be displayed as their own file. If opened in the app, just the setup guide will be displayed on the right. Thanks for clarifying!

The issue I see with this approach is that our current setup guides in the documentation is not very helpful, and refer other links if you want to know more. If we want to go this way, I feel we need to make the setup guides more explicit (possibly with screenshots). Wdyt @avaidyanatha ?

@johnlafleur - I absolutely agree that our setup guides will need to be more explicit (which is the direction they should go anyway) for this approach. But I think as a POC we can have a guide that simply explains each config input to get us going if writing explicit guides for each one takes too long.

[avaidyanatha]

@cgardens and I went over this - in terms of retrieving the getting-started.md file, we would attempt to pull the latest version from GitHub directly and if an internet connection cannot be established, we fall back to a static, and potentially out of date version accessible locally.

[cgardens]

I lost track of this issue because it wasn't labelled as frontend. Properly labelled now.

Technical details:

• We will need to figure out how to get all of the docs onto the server docker image
• If possible we should try to fetch new docs (as abhi mentions) by hit github directly for the markdown file. If it is not present then fall back on the local version. I'd say for now we can just have the FE hit GH directly, we can move it behind the API later if we want to.
• FE will need a markdown reader that handle the custom markdown that we use for gitbook.

@avaidyanatha were you planning to split up the docs for each connector yourself into the 3 component parts? or should that be part of this issue as well?

[johnlafleur]

To be honest, I'm still a bit torn on that issue. I'm wondering if OAuth would just fix the initial problem, and if we shouldn't wait to have Oauth before deciding whether we want to spend time on this.

[avaidyanatha]

@avaidyanatha were you planning to split up the docs for each connector yourself into the 3 component parts? or should that be part of this issue as well?

Yep, I was planning on doing this, (hopefully with the help of @marcosmarxm) but it should still be part of the issue.