🎉 Source Okta: added parameter 'start_date' #15050

drrest · 2022-07-26T21:50:24Z

What

added: parameter 'start_date' to source Okta

How

Added filed in request parameters.

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user? If yes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.
/test connector=connectors/<name> command is passing
New Connector version released on Dockerhub and connector version bumped by running the /publish command described here

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

added: parameter 'start_date' to source Okta changed: unit tests

drrest · 2022-07-26T21:51:33Z

/test connector=connectors/source-okta

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742745203
❌ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742745203
🐛 https://gradle.com/s/u7ca3a7gwphy4

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
=================== 1 failed, 30 passed in 82.51s (0:01:22) ====================

drrest · 2022-07-26T22:17:41Z

/test connector=connectors/source-okta

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742864137
❌ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2742864137
🐛 https://gradle.com/s/olm6wpxnbnxn2

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestDiscovery::test_additional_properties_is_true[inputs0]
=================== 1 failed, 30 passed in 86.55s (0:01:26) ====================

airbyte-integrations/connectors/source-okta/source_okta/source.py

…art_date_parameter_to_okta_connector

drrest · 2022-07-29T11:59:09Z

/test connector=connectors/source-okta

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2760467751
✅ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2760467751
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        77      6    92%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/tests/test_incremental.py       121     25    79%
source_acceptance_test/utils/common.py                  77     17    78%
source_acceptance_test/tests/test_core.py              307    106    65%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
------------------------------------------------------------------------
TOTAL                                                  970    246    75%
Name                      Stmts   Miss  Cover
---------------------------------------------
source_okta/__init__.py       2      0   100%
source_okta/source.py       189      1    99%
---------------------------------------------
TOTAL                       191      1    99%

Build Passed

Test summary info:

All Passed

bazarnov · 2022-07-30T18:07:22Z

Please make ./gradlew clean format after all your work has been done and commit those changes as well because this CI job fails due to bad formatting, optionally you can merge with master to pick up latest changes before merging into master.

…art_date_parameter_to_okta_connector

…kta_connector

airbyte-integrations/connectors/source-okta/source_okta/source.py

drrest · 2022-08-03T19:33:01Z

/publish connector=connectors/source-okta

🕑 Publishing the following connectors:
connectors/source-okta
https://github.com/airbytehq/airbyte/actions/runs/2791973155

Connector	Did it publish?	Were definitions generated?
connectors/source-okta	❌	❌

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

airbyte-integrations/connectors/source-okta/source_okta/source.py

lazebnyi · 2022-08-05T23:04:23Z

/test connector=connectors/source-okta

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806709455
❌ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806709455
🐛 https://gradle.com/s/iwjfobjxmuima

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_token_authentication_init
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_oauth2_authentication_init
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_oauth2_authentication_wrong_credentials_record
	 FAILED unit_tests/test_source.py::TestAuthentication::test_init_oauth2_authentication_wrong_oauth_config_bad_auth_type
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_connection_ok
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_connection_error_status_code
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_connection_error_with_exception
	 FAILED unit_tests/test_source.py::TestAuthentication::test_check_streams - At...
	 FAILED unit_tests/test_source.py::TestAuthentication::test_oauth2_refresh_token_ok
	 FAILED unit_tests/test_source.py::TestAuthentication::test_oauth2_refresh_token_failed
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.OK-False]
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.BAD_REQUEST-False]
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.TOO_MANY_REQUESTS-True]
	 FAILED unit_tests/test_streams.py::TestStatusCodes::test_should_retry[HTTPStatus.INTERNAL_SERVER_ERROR-True]
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_request_params
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_parse_response
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_backoff_time
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_request_params
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_incremental_okta_stream_parse_response
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_incremental_okta_stream_backoff_time
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_backoff_time_empty
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_back_off_now
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_incremental_get_updated_state
	 FAILED unit_tests/test_streams.py::TestOktaStream::test_okta_stream_http_method
	 FAILED unit_tests/test_streams.py::TestNextPageToken::test_next_page_token - ...
	 FAILED unit_tests/test_streams.py::TestNextPageToken::test_next_page_token_empty_params
	 FAILED unit_tests/test_streams.py::TestNextPageToken::test_next_page_token_link_have_self_and_equal_next
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_stream_users - TypeE...
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_request_params_out_of_next_page_token
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_source_request_params_have_next_cursor
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_source_request_params_have_latest_entry
	 FAILED unit_tests/test_streams.py::TestStreamUsers::test_users_source_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamCustomRoles::test_custom_roles
	 FAILED unit_tests/test_streams.py::TestStreamCustomRoles::test_custom_roles_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamPermissions::test_permissions - ...
	 FAILED unit_tests/test_streams.py::TestStreamPermissions::test_permissions_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroups::test_groups - TypeError:...
	 FAILED unit_tests/test_streams.py::TestStreamGroups::test_groups_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members_request_params_with_latest_entry
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_members_slice_stream
	 FAILED unit_tests/test_streams.py::TestStreamGroupMembers::test_group_member_request_get_update_state
	 FAILED unit_tests/test_streams.py::TestStreamGroupRoleAssignment::test_group_role_assignments
	 FAILED unit_tests/test_streams.py::TestStreamGroupRoleAssignment::test_group_role_assignments_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamGroupRoleAssignment::test_group_role_assignments_slice_stream
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs - TypeError: __i...
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs_request_params_for_since
	 FAILED unit_tests/test_streams.py::TestStreamLogs::test_logs_request_params_for_until
	 FAILED unit_tests/test_streams.py::TestStreamUserRoleAssignment::test_user_role_assignments
	 FAILED unit_tests/test_streams.py::TestStreamUserRoleAssignment::test_user_role_assignments_parse_response
	 FAILED unit_tests/test_streams.py::TestStreamUserRoleAssignment::test_user_role_assignments_slice_stream
	 FAILED unit_tests/test_utils.py::test_get_start_date[config2-expected_date2-expected_messages2]
	 �[31m========================= �[31m�[1m54 failed�[0m, �[32m2 passed�[0m�[31m in 1.51s�[0m�[31m =========================�[0m

lazebnyi · 2022-08-05T23:43:01Z

/test connector=connectors/source-okta

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515
❌ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515
🐛 https://gradle.com/s/oeakgteownrkm

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs0] - AssertionError: All ...
FAILED test_core.py::TestBasicRead::test_read[inputs1] - AssertionError: All ...
=================== 2 failed, 34 passed in 89.60s (0:01:29) ====================

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515
❌ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515
🐛 https://gradle.com/s/ok52jqgzbe6gm

Build Failed

Test summary info:

=========================== short test summary info ============================
FAILED test_core.py::TestBasicRead::test_read[inputs1] - AssertionError: All ...
FAILED test_full_refresh.py::TestFullRefresh::test_sequential_reads[inputs0]
=================== 2 failed, 34 passed in 220.11s (0:03:40) ===================

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515
❌ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2806843515
🐛 https://gradle.com/s/ysy636cvvkr7c

Build Failed

Test summary info:

	 =========================== short test summary info ============================
	 FAILED unit_tests/test_utils.py::test_get_start_date[config1-expected_date1-expected_messages1]
	 FAILED unit_tests/test_utils.py::test_get_start_date[config2-expected_date2-expected_messages2]
	 �[31m================== �[31m�[1m2 failed�[0m, �[32m54 passed�[0m, �[33m132 warnings�[0m�[31m in 0.71s�[0m�[31m ==================�[0m

lazebnyi · 2022-08-06T01:35:34Z

/test connector=connectors/source-okta

🕑 connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2807174148
✅ connectors/source-okta https://github.com/airbytehq/airbyte/actions/runs/2807174148
Python tests coverage:

Name                                                 Stmts   Miss  Cover
------------------------------------------------------------------------
source_acceptance_test/utils/__init__.py                 6      0   100%
source_acceptance_test/tests/__init__.py                 4      0   100%
source_acceptance_test/__init__.py                       2      0   100%
source_acceptance_test/tests/test_full_refresh.py       52      2    96%
source_acceptance_test/utils/asserts.py                 37      2    95%
source_acceptance_test/config.py                        82      6    93%
source_acceptance_test/utils/json_schema_helper.py     105     13    88%
source_acceptance_test/tests/test_incremental.py       121     25    79%
source_acceptance_test/utils/common.py                  77     17    78%
source_acceptance_test/tests/test_core.py              355    107    70%
source_acceptance_test/utils/compare.py                 62     23    63%
source_acceptance_test/base.py                          10      4    60%
source_acceptance_test/utils/connector_runner.py       110     48    56%
------------------------------------------------------------------------
TOTAL                                                 1023    247    76%
Name                           Stmts   Miss  Cover
--------------------------------------------------
source_okta/utils.py              44      0   100%
source_okta/authenticator.py      14      0   100%
source_okta/__init__.py            2      0   100%
source_okta/source.py            174      7    96%
--------------------------------------------------
TOTAL                            234      7    97%

Build Passed

Test summary info:

All Passed

lazebnyi · 2022-08-08T08:59:45Z

/publish connector=connectors/source-okta

🕑 Publishing the following connectors:
connectors/source-okta
https://github.com/airbytehq/airbyte/actions/runs/2816679958

Connector	Did it publish?	Were definitions generated?
connectors/source-okta	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

Added parameter 'start_date' in Okta source

756d921

added: parameter 'start_date' to source Okta changed: unit tests

drrest requested a review from bazarnov July 26, 2022 21:50

github-actions bot added the area/connectors Connector related issues label Jul 26, 2022

octavia-squidington-iii added the connectors/source/okta label Jul 26, 2022

changes: fix in the case of ISSUE: #14196

e67b14b

drrest linked an issue Jul 26, 2022 that may be closed by this pull request

Add start_date parameter to Okta connector #10375

Closed

bazarnov suggested changes Jul 27, 2022

View reviewed changes

airbyte-integrations/connectors/source-okta/source_okta/source.py Outdated Show resolved Hide resolved

Merge remote-tracking branch 'origin/master' into drrest/10375-add_st…

5893e03

…art_date_parameter_to_okta_connector

Okta documentation in new format

ddcb086

github-actions bot added the area/documentation Improvements or additions to documentation label Jul 29, 2022

drrest requested review from bazarnov and lazebnyi July 29, 2022 13:41

drrest added 2 commits July 29, 2022 18:03

changes: fix to use super() instead of instance of stream parent

85f6991

changes: additional changes into OKTA documentaton

534b92e

bazarnov approved these changes Jul 29, 2022

View reviewed changes

drrest added 2 commits August 2, 2022 13:53

Merge remote-tracking branch 'origin/master' into drrest/10375-add_st…

2be9e69

…art_date_parameter_to_okta_connector

changes: switch release to beta

e4efe0f

drrest temporarily deployed to more-secrets August 2, 2022 12:39 Inactive

Merge branch 'master' into drrest/10375-add_start_date_parameter_to_o…

2588ee0

…kta_connector

drrest temporarily deployed to more-secrets August 3, 2022 08:20 Inactive

lazebnyi approved these changes Aug 3, 2022

View reviewed changes

drrest requested a review from girarda August 3, 2022 11:38

sherifnada approved these changes Aug 3, 2022

View reviewed changes

sherifnada removed the request for review from girarda August 3, 2022 17:01

sherifnada reviewed Aug 3, 2022

View reviewed changes

airbyte-integrations/connectors/source-okta/source_okta/source.py Outdated Show resolved Hide resolved

drrest temporarily deployed to more-secrets August 3, 2022 20:18 Inactive

Merged master to branch

4d24302

lazebnyi temporarily deployed to more-secrets August 5, 2022 17:19 Inactive

Rollback releaseStage

eebfb88

lazebnyi temporarily deployed to more-secrets August 5, 2022 17:26 Inactive

Refactored start date logic

0398c45

lazebnyi requested a review from sherifnada August 5, 2022 20:00

lazebnyi temporarily deployed to more-secrets August 5, 2022 20:01 Inactive

sherifnada suggested changes Aug 5, 2022

View reviewed changes

airbyte-integrations/connectors/source-okta/source_okta/source.py Outdated Show resolved Hide resolved

Deleted microseconds from state

f53899e

lazebnyi temporarily deployed to more-secrets August 5, 2022 21:55 Inactive

Add start date to all streams

d83d1c1

lazebnyi temporarily deployed to more-secrets August 5, 2022 22:55 Inactive

lazebnyi requested a review from sherifnada August 5, 2022 22:58

Updated to linter

f43ad2e

lazebnyi temporarily deployed to more-secrets August 5, 2022 23:06 Inactive

sherifnada approved these changes Aug 5, 2022

View reviewed changes

Fixed unit tests

7630760

lazebnyi temporarily deployed to more-secrets August 5, 2022 23:44 Inactive

Updated unit tests

fb14ffd

lazebnyi temporarily deployed to more-secrets August 6, 2022 01:37 Inactive

auto-bump connector version [ci skip]

c8c8bb3

lazebnyi merged commit 8d9a3aa into master Aug 8, 2022

lazebnyi deleted the drrest/10375-add_start_date_parameter_to_okta_connector branch August 8, 2022 10:00

octavia-squidington-iii mentioned this pull request Aug 10, 2022

Bump Airbyte version from 0.39.42-alpha to 0.40.0-alpha #15493

Merged

YiyangLi mentioned this pull request Aug 10, 2022

Source Okta: add resource-sets (incremental supported) #14700

Merged

37 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🎉 Source Okta: added parameter 'start_date' #15050

🎉 Source Okta: added parameter 'start_date' #15050

Uh oh!

drrest commented Jul 26, 2022

Uh oh!

drrest commented Jul 26, 2022 •

edited by github-actions bot

Loading

Uh oh!

drrest commented Jul 26, 2022 •

edited by github-actions bot

Loading

Uh oh!

Uh oh!

drrest commented Jul 29, 2022 •

edited by github-actions bot

Loading

Uh oh!

bazarnov commented Jul 30, 2022 •

edited

Loading

Uh oh!

Uh oh!

drrest commented Aug 3, 2022 •

edited by github-actions bot

Loading

Uh oh!

Uh oh!

lazebnyi commented Aug 5, 2022 •

edited by github-actions bot

Loading

Uh oh!

lazebnyi commented Aug 5, 2022 •

edited by github-actions bot

Loading

Uh oh!

lazebnyi commented Aug 6, 2022 •

edited by github-actions bot

Loading

Uh oh!

lazebnyi commented Aug 8, 2022 •

edited by github-actions bot

Loading

Uh oh!

Uh oh!

🎉 Source Okta: added parameter 'start_date' #15050

🎉 Source Okta: added parameter 'start_date' #15050

Uh oh!

Conversation

drrest commented Jul 26, 2022

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Checklist

Airbyter

Tests

Uh oh!

drrest commented Jul 26, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

drrest commented Jul 26, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

Uh oh!

drrest commented Jul 29, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

bazarnov commented Jul 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

drrest commented Aug 3, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

lazebnyi commented Aug 5, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Uh oh!

lazebnyi commented Aug 5, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Failed

Build Failed

Build Failed

Uh oh!

lazebnyi commented Aug 6, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

lazebnyi commented Aug 8, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

drrest commented Jul 26, 2022 •

edited by github-actions bot

Loading

drrest commented Jul 26, 2022 •

edited by github-actions bot

Loading

drrest commented Jul 29, 2022 •

edited by github-actions bot

Loading

bazarnov commented Jul 30, 2022 •

edited

Loading

drrest commented Aug 3, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Aug 5, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Aug 5, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Aug 6, 2022 •

edited by github-actions bot

Loading

lazebnyi commented Aug 8, 2022 •

edited by github-actions bot

Loading