Skip to content

Re-write sonic-db-cli with c++ for sonic startup performance issue #10491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from

Conversation

liuh-80
Copy link
Contributor

@liuh-80 liuh-80 commented Apr 7, 2022

Why I did it

Fix sonic-db-cli high CPU usage on SONiC startup issue: https://github.com/Azure/sonic-buildimage/issues/10218

How I did it

Re-write sonic-cli with c++.

How to verify it

Add c++ unit test to cover all code.
Pass all E2E test scenario.

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111

Description for the changelog

Re-write sonic-cli with c++ for sonic startup performance issue

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

@liuh-80 liuh-80 changed the title Re-write sonic-cli with c++ for sonic startup performance issue Re-write sonic-db-cli with c++ for sonic startup performance issue Apr 7, 2022
AUTOMAKE_OPTIONS = subdir-objects
SUBDIRS = tests

bin_PROGRAMS = sonic-db-cli
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sonic-db-cli

Offline discussed. Let's move this application to sonic-swss-common repo. Following original practice that in sonic-py-swsssdk repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will close this PR and send another PR for this.

bool use_unix_socket)
{
std::unique_ptr<swss::SonicV2Connector_Native> dbconn;
if (name_space.compare("None") == 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{

The code style is not consistent in one file. Please try follow the existing code in sonic-swss-common repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix this in another PR.

{
std::unique_ptr<swss::SonicV2Connector_Native> dbconn;
if (name_space.compare("None") == 0) {
dbconn = std::make_unique<swss::SonicV2Connector_Native>(use_unix_socket, empty_str);
Copy link
Collaborator

@qiluo-msft qiluo-msft Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SonicV2Connector_Native

The classes of SonicV2Connector* are only used by SWIG and then by python application. The equal class in C++ is DBConnector.

You may check SonicV2Connector_Native implementation to find the equivalent DBConnector functions.

bool use_unix_socket)
{
std::unique_ptr<swss::SonicV2Connector_Native> dbconn;
if (name_space.compare("None") == 0) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compare

Just use ==?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix in another PR.

auto& client = dbconn->get_redis_client(db_name);

size_t argc = commands.size();
const char** argv = new const char*[argc];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new

Memory leak.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix in another PR.


size_t argc = commands.size();
const char** argv = new const char*[argc];
size_t* argvc = new size_t[argc];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new

Memory leak.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix in another PR.

size_t* argvc = new size_t[argc];
for (size_t i = 0; i < argc; i++)
{
argv[i] = strdup(commands[i].c_str());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strdup

Memory leak.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will rewrite this method in another PR, because there are so many memory leak.

switch(reply->type)
{
case REDIS_REPLY_INTEGER:
std::cout << reply->integer;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

std::cout

Is printf faster?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

void handleAllInstances(
const std::string& name_space,
Copy link
Collaborator

@qiluo-msft qiluo-msft Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name_space

We name it like netns, ns_name or m_namespace as in other c++ code in sonic-swss-common repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, will submit in another PR, because this file will check-in to sonic-swss-common repo.

void handleAllInstances(
const std::string& name_space,
const std::string& operation,
bool use_unix_socket)
Copy link
Collaborator

@qiluo-msft qiluo-msft Apr 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use_unix_socket

If this is a pure-C++ application, not to used by SWIG/python at all, we prefer camelCaseName, and try follow existing code as much as possible.

For this parameter, please check isTcpConn.

This is a general comment applied to all the source code files, all variable names, all functions names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, will update in another PR.

@@ -0,0 +1,15 @@
#INCLUDES = -I $(top_srcdir)

bin_PROGRAMS = tests
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests

Discussed offline. Let's move testcase into sonic-swss-common repo.

You can use real redis instances in unit test.
You can use either googletest or pytest frameworks.
You can use sonic-db-cli as a process.

I am thinking about reuse the same set of testcases in order to test the behavior of old python version of sonic-db-cli. The co-test could last for a while until the python version is completely deprecated and removed from all applications.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIll also move sests to sonic-swss-common and submit in another PR.

@liuh-80
Copy link
Contributor Author

liuh-80 commented Apr 24, 2022

Close this PR and create another PR in sonic-swss-common repo: sonic-net/sonic-swss-common#607

@liuh-80 liuh-80 closed this Apr 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants