fix: Create your own source connector tutorial (#21985)

* First commit

* Fix some required parts and added extra information.
two tests failing:
```
       2 failed
         - ../actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_core.py:699 TestBasicRead.test_airbyte_trace_message_on_failure[inputs0]
         - ../actions-runner/_work/airbyte/airbyte/airbyte-integrations/bases/source-acceptance-test/source_acceptance_test/tests/test_incremental.py:196 TestIncremental.test_read_sequential_slices[inputs0]
```

* Improve message

* Remove copy-paste line :D

Author: sergio-ropero

docs/connector-development/tutorials/adding-incremental-sync.md

@@ -37,6 +37,38 @@ def discover():
     print(json.dumps(airbyte_message))
 ```
 
+Also, create a file called `incremental_configured_catalog.json` with the following content:
+```javascript
+{
+    "streams": [
+        {
+            "stream": {
+                "name": "stock_prices",
+                "supported_sync_modes": [
+                    "full_refresh",
+                    "incremental"
+                ],
+                "json_schema": {
+                    "properties": {
+                        "date": {
+                            "type": "string"
+                        },
+                        "price": {
+                            "type": "number"
+                        },
+                        "stock_ticker": {
+                            "type": "string"
+                        }
+                    }
+                }
+            },
+            "sync_mode": "full_refresh",
+            "destination_sync_mode": "overwrite"
+        }
+    ]
+}
+```
+
 ## Update `read`
 
 Next we will adapt the `read` method that we wrote previously. We need to change three things.
@@ -63,7 +95,7 @@ Here's what our updated `read` method would look like.
 def read(config, catalog, state):
     # Assert required configuration was provided
     if "api_key" not in config or "stock_ticker" not in config:
-        log("Input config must contain the properties 'api_key' and 'stock_ticker'")
+        log_error("Input config must contain the properties 'api_key' and 'stock_ticker'")
         sys.exit(1)
 
     # Find the stock_prices stream if it is present in the input catalog
@@ -73,7 +105,7 @@ def read(config, catalog, state):
             stock_prices_stream = configured_stream
 
     if stock_prices_stream is None:
-        log("No streams selected")
+        log_error("No streams selected")
         return
 
     # By default we fetch stock prices for the 7 day period ending with today
@@ -93,7 +125,7 @@ def read(config, catalog, state):
         response = _call_api(ticker=config["stock_ticker"], token = config["api_key"], from_day=from_day, to_day=cursor_value)
         if response.status_code != 200:
             # In a real scenario we'd handle this error better :)
-            log("Failure occurred when calling Polygon.io API")
+            log_error("Failure occurred when calling Polygon.io API")
             sys.exit(1)
         else:
             # Stock prices are returned sorted by date in ascending order
@@ -117,6 +149,11 @@ def read(config, catalog, state):
         print(json.dumps(output_message))
 ```
 
+That code requires to add a new library import in the `source.py` file:
+```python
+from datetime import timezone
+```
+
 We will also need to parse `state` argument in the `run` method. In order to do that, we will modify the code that 
 calls `read` method from `run` method:
 ```python
@@ -209,6 +246,23 @@ You will also need to create an `abnormal_state.json` file with a date in the fu
 {"stock_prices": {"date": "2121-01-01"}}
 ```
 
+And lastly you need to modify the `check` function call to include the new parameters `from_day` and `to_day` in `source.py`:
+```python
+def check(config):
+    # Validate input configuration by attempting to get the daily closing prices of the input stock ticker
+    response = _call_api(ticker=config["stock_ticker"], token=config["api_key"], from_day=datetime.now().date()-timedelta(days=1), to_day=datetime.now().date())
+    if response.status_code == 200:
+        result = {"status": "SUCCEEDED"}
+    elif response.status_code == 403:
+        # HTTP code 403 means authorization failed so the API key is incorrect
+        result = {"status": "FAILED", "message": "API Key is incorrect."}
+    else:
+        result = {"status": "FAILED", "message": "Input configuration is incorrect. Please verify the input stock ticker and API key."}
+
+    output_message = {"type": "CONNECTION_STATUS", "connectionStatus": result}
+    print(json.dumps(output_message))
+```
+
 Run the tests once again:
 
 ```bash

docs/connector-development/tutorials/build-a-connector-the-hard-way.md

@@ -33,6 +33,11 @@ Python 3.9.11
 
 On some systems, `python` points to a Python2 installation and `python3` points to Python3. If this is the case on your machine, substitute all `python` commands in this guide with `python3` . Otherwise, make sure to install Python 3 before beginning.
 
+You need also to install `requests` python library:
+````bash
+pip install requests 
+````
+
 ## Our connector: a stock ticker API
 
 Our connector will output the daily price of a stock since a given date. We'll leverage the free [Polygon.io API](https://polygon.io/pricing) for this. We'll use Python to implement the connector because its syntax is accessible to most programmers, but the process described here can be applied to any language.
@@ -208,7 +213,7 @@ def run(args):
     else:
         # If we don't recognize the command log the problem and exit with an error code greater than 0 to indicate the process
         # had a failure
-        log("Invalid command. Allowable commands: [spec]")
+        log_error("Invalid command. Allowable commands: [spec]")
         sys.exit(1)
 
     # A zero exit code means the process successfully completed
@@ -228,6 +233,8 @@ Some notes on the above code:
 
 1. As described in the [specification](https://docs.airbyte.com/understanding-airbyte/airbyte-protocol/#key-takeaways), Airbyte connectors are CLIs which communicate via stdout, so the output of the command is simply a JSON string formatted according to the Airbyte Specification. So to "return" a value we use `print` to output the return value to stdout
 2. All Airbyte commands can output log messages that take the form `{"type":"LOG", "log":"message"}`, so we create a helper method `log(message)` to allow logging
+3. All Airbyte commands can output error messages that take the form `{"type":"TRACE", "trace": {"type": "ERROR", "emitted_at": current_time_in_ms, "error": {"message": error_message}}}}`, so we create a helper method `log_error(message)` to allow error messages
+
 
 Now if we run `python source.py spec` we should see the specification printed out:
 
@@ -262,8 +269,8 @@ Then we'll add the `check_method`:
 
 ```python
 import requests
-import datetime
 from datetime import date
+from datetime import datetime
 from datetime import timedelta
 
 def _call_api(ticker, token):
@@ -352,7 +359,7 @@ def run(args):
     else:
         # If we don't recognize the command log the problem and exit with an error code greater than 0 to indicate the process
         # had a failure
-        log("Invalid command. Allowable commands: [spec, check]")
+        log_error("Invalid command. Allowable commands: [spec, check]")
         sys.exit(1)
 
     # A zero exit code means the process successfully completed
@@ -471,7 +478,7 @@ def run(args):
     else:
         # If we don't recognize the command log the problem and exit with an error code greater than 0 to indicate the process
         # had a failure
-        log("Invalid command. Allowable commands: [spec, check, discover]")
+        log_error("Invalid command. Allowable commands: [spec, check, discover]")
         sys.exit(1)
 
     # A zero exit code means the process successfully completed
@@ -545,7 +552,7 @@ Then we'll define the `read` method in `source.py`:
 
 ```python
 def log_error(error_message):
-    current_time_in_ms = int(datetime.datetime.now().timestamp()) * 1000
+    current_time_in_ms = int(datetime.now().timestamp()) * 1000
     log_json = {"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": current_time_in_ms, "error": {"message": error_message}}}
     print(json.dumps(log_json))
 
@@ -583,7 +590,7 @@ def read(config, catalog):
         results = response.json()["results"]
         for result in results:
             data = {"date": date.fromtimestamp(result["t"]/1000).isoformat(), "stock_ticker": config["stock_ticker"], "price": result["c"]}
-            record = {"stream": "stock_prices", "data": data, "emitted_at": int(datetime.datetime.now().timestamp()) * 1000}
+            record = {"stream": "stock_prices", "data": data, "emitted_at": int(datetime.now().timestamp()) * 1000}
             output_message = {"type": "RECORD", "record": record}
             print(json.dumps(output_message))
 ```
@@ -615,7 +622,7 @@ elif command == "read":
 and: 
 
 ```python
-        log("Invalid command. Allowable commands: [spec, check, discover, read]")
+        log_error("Invalid command. Allowable commands: [spec, check, discover, read]")
 ```
 
 this yields the following `run` method:
@@ -666,7 +673,7 @@ def run(args):
     else:
         # If we don't recognize the command log the problem and exit with an error code greater than 0 to indicate the process
         # had a failure
-        log("Invalid command. Allowable commands: [spec, check, discover, read]")
+        log_error("Invalid command. Allowable commands: [spec, check, discover, read]")
         sys.exit(1)
 
     # A zero exit code means the process successfully completed
@@ -717,8 +724,8 @@ import json
 import sys
 import os
 import requests
-import datetime
 from datetime import date
+from datetime import datetime
 from datetime import timedelta
 
 def read(config, catalog):
@@ -754,7 +761,7 @@ def read(config, catalog):
         results = response.json()["results"]
         for result in results:
             data = {"date": date.fromtimestamp(result["t"]/1000).isoformat(), "stock_ticker": config["stock_ticker"], "price": result["c"]}
-            record = {"stream": "stock_prices", "data": data, "emitted_at": int(datetime.datetime.now().timestamp()) * 1000}
+            record = {"stream": "stock_prices", "data": data, "emitted_at": int(datetime.now().timestamp()) * 1000}
             output_message = {"type": "RECORD", "record": record}
             print(json.dumps(output_message))
 
@@ -799,7 +806,7 @@ def log(message):
 
 
 def log_error(error_message):
-   current_time_in_ms = int(datetime.datetime.now().timestamp()) * 1000
+   current_time_in_ms = int(datetime.now().timestamp()) * 1000
    log_json = {"type": "TRACE", "trace": {"type": "ERROR", "emitted_at": current_time_in_ms, "error": {"message": error_message}}}
    print(json.dumps(log_json))
 
@@ -892,7 +899,7 @@ def run(args):
     else:
         # If we don't recognize the command log the problem and exit with an error code greater than 0 to indicate the process
         # had a failure
-        log("Invalid command. Allowable commands: [spec, check, discover, read]")
+        log_error("Invalid command. Allowable commands: [spec, check, discover, read]")
         sys.exit(1)
 
     # A zero exit code means the process successfully completed