Skip to content

Support for SageMaker Notebook Operators #32702

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
1 of 2 tasks
shubham22 opened this issue Jul 19, 2023 · 5 comments · Fixed by #33219
Closed
1 of 2 tasks

Support for SageMaker Notebook Operators #32702

shubham22 opened this issue Jul 19, 2023 · 5 comments · Fixed by #33219
Assignees
@ellisms
Labels
area:providers good first issue kind:feature Feature Requests provider:amazon AWS/Amazon - related issues

Comments

@shubham22
Copy link

Description

Today, Amazon provider package supports SageMaker operators for a few operations, like training, tuning, pipelines, but it lacks the support for SageMaker Notebook instances. Boto3 provides necessary APIs to create notebook instance, start notebook instance, stop network instance and delete notebook instance. Leveraging these APIs, we should add new operators to SageMaker set under Amazon provider. At the same time, having a sensor (synchronous as well as deferrable) for notebook instance execution that utilizes describe notebook instance and waits for Stopped/Failed status would help with observability of the execution.

Use case/motivation

Data Scientists are orchestrating ML use cases via Apache Airflow. A key component of ML use cases is running Jupyter Notebook on SageMaker. Having built-in operators and sensors would make it easy for Airflow users to run Notebook instances on SageMaker.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@shubham22 shubham22 added kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet labels Jul 19, 2023
@boring-cyborg
Copy link

boring-cyborg bot commented Jul 19, 2023

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

@eladkal eladkal added provider:amazon AWS/Amazon - related issues area:providers good first issue and removed needs-triage label for new issues that we didn't triage yet labels Jul 19, 2023
@ellisms
Copy link
Contributor

ellisms commented Jul 27, 2023

I'd be interested in taking this one, if it is still available.

@eladkal
Copy link
Contributor

eladkal commented Jul 27, 2023

@ellisms assigned

@ShamithaReddy
Copy link

hey @ellisms may i know if you are still working on this issue please.

@eladkal
Copy link
Contributor

eladkal commented Aug 12, 2023

hey @ellisms may i know if you are still working on this issue please.

PR: #33219

@eladkal eladkal mentioned this issue Aug 12, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ellisms ellisms

Labels
area:providers good first issue kind:feature Feature Requests provider:amazon AWS/Amazon - related issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Added Amazon SageMaker Notebook hook and operators ellisms/airflow
4 participants
@eladkal @ShamithaReddy @shubham22 @ellisms