Skip to content

fix(report): corrections to report #526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 22, 2019
Merged

fix(report): corrections to report #526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2fc4bd3
fix(report): corrections to report
May 28, 2019
fb70f43
fix(report): Fix merge error, fix rest payload
lepokle Jul 19, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ...d/src/src-licenseinfo/src/main/java/org/eclipse/sw360/licenseinfo/LicenseInfoHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ protected LicenseInfoHandler(AttachmentDatabaseHandler attachmentDatabaseHandler
new TextGenerator(DISCLOSURE, "License Disclosure as TEXT"),
new XhtmlGenerator(DISCLOSURE, "License Disclosure as XHTML"),
new DocxGenerator(DISCLOSURE, "License Disclosure as DOCX"),
new DocxGenerator(REPORT, "License Report as DOCX")
new DocxGenerator(REPORT, "Project Clearing Report as DOCX")
);
// @formatter:on
}
Expand Down
99 changes: 68 additions & 31 deletions ...censeinfo/src/main/java/org/eclipse/sw360/licenseinfo/outputGenerators/DocxGenerator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import org.apache.log4j.Logger;
import org.apache.poi.xwpf.usermodel.*;
import org.apache.thrift.TException;
import org.apache.xmlbeans.XmlCursor;
import org.apache.xmlbeans.XmlException;
import org.eclipse.sw360.datahandler.common.CommonUtils;
import org.eclipse.sw360.datahandler.common.SW360Utils;
Expand All @@ -33,6 +34,8 @@
import org.eclipse.sw360.datahandler.thrift.licenses.LicenseService;
import org.eclipse.sw360.datahandler.thrift.licenses.Todo;
import org.eclipse.sw360.datahandler.thrift.projects.Project;
import org.eclipse.sw360.datahandler.thrift.projects.ProjectLink;
import org.eclipse.sw360.datahandler.thrift.projects.ProjectService;
import org.eclipse.sw360.datahandler.thrift.users.User;
import org.eclipse.sw360.datahandler.thrift.users.UserService;

Expand All @@ -51,17 +54,24 @@ public class DocxGenerator extends OutputGenerator<byte[]> {
private static final Logger LOGGER = Logger.getLogger(DocxGenerator.class);
private static final String UNKNOWN_LICENSE_NAME = "Unknown license name";
private static final String UNKNOWN_FILE_NAME = "Unknown file name";
private static final String UNKNOWN_LICENSE = "Unknown";
private static final String TODO_DEFAULT_TEXT = "todo not determined so far.";

private static final String DOCX_TEMPLATE_FILE = "/templateFrontpageContent.docx";
private static final String DOCX_TEMPLATE_REPORT_FILE = "/templateReport.docx";
private static final String DOCX_MIME_TYPE = "application/vnd.openxmlformats-officedocument.wordprocessingml.document";
private static final String DOCX_OUTPUT_TYPE = "docx";
public static final String UNKNOWN_LICENSE = "Unknown";

private static final long ADDITIONAL_REQ_THRESHOLD = 3;
public static final int ADDITIONAL_REQ_TABLE_INDEX = 5;

public static final int OVERVIEW_TABLE_INDEX = 0;
public static final int SPECIAL_OSS_RISKS_TABLE_INDEX = 1;
public static final int DEV_DETAIL_TABLE_INDEX = 2;
public static final int THIRD_PARTY_COMPONENT_OVERVIEW_TABLE_INDEX = 3;
private static final int COMMON_RULES_TABLE_INDEX = 4;
public static final int ADDITIONAL_REQ_TABLE_INDEX = 5;



public DocxGenerator(OutputFormatVariant outputFormatVariant, String description) {
super(DOCX_OUTPUT_TYPE, description, true, DOCX_MIME_TYPE, outputFormatVariant);
Expand Down Expand Up @@ -140,7 +150,7 @@ private void fillDisclosureDocument(
fillReleaseBulletList(document, projectLicenseInfoResults);
fillReleaseDetailList(document, projectLicenseInfoResults, includeObligations);
fillLicenseList(document, projectLicenseInfoResults);
}
}

private void fillReportDocument(
XWPFDocument document,
Expand All @@ -151,6 +161,7 @@ private void fillReportDocument(
Collection<ObligationParsingResult> obligationResults,
User user) throws XmlException, TException {

String businessUnit = project.getBusinessUnit();
String projectName = project.getName();
String projectVersion = project.getVersion();
String obligationsText = project.getObligationsText();
Expand All @@ -161,11 +172,13 @@ private void fillReportDocument(
String deliveryChannelsText = project.getDeliveryChannels();
String remarksAdditionalRequirementsText = project.getRemarksAdditionalRequirements();
String projectDescription = project.getDescription();

// extract licenses that appear at least ADDITIONAL_REQ_THRESHOLD times
Set<String> mostLicenses = extractMostCommonLicenses(obligationResults, ADDITIONAL_REQ_THRESHOLD);

fillOwnerGroup(document, project);
fillAttendeesTable(document, project);

replaceText(document, "$bunit", businessUnit);
replaceText(document, "$license-info-header", licenseInfoHeaderText);
replaceText(document, "$project-name", projectName);
replaceText(document, "$project-version", projectVersion);
Expand All @@ -179,10 +192,12 @@ private void fillReportDocument(
replaceText(document, "$product-description", projectDescription);

fillSpecialOSSRisksTable(document, project, obligationResults);
fillDevelopmentDetailsTable(document, project, user);
fillDevelopmentDetailsTable(document, project, user, projectLicenseInfoResults);
fillOverview3rdPartyComponentTable(document, projectLicenseInfoResults);

fillCommonRulesTable(document, project);
fillAdditionalRequirementsTable(document, obligationResults);
replaceText(document, "$list_comma_sep_licenses_above_threshold", String.join(", ", mostLicenses));
fillAdditionalRequirementsTable(document, obligationResults, mostLicenses);

fillCommonRulesTable(document, project);

Expand All @@ -200,9 +215,9 @@ private void fillOwnerGroup(XWPFDocument document, Project project) throws XmlEx
}

private void fillAttendeesTable(XWPFDocument document, Project project) throws XmlException, TException {
XWPFTable table = document.getTables().get(0);
XWPFTable table = document.getTables().get(OVERVIEW_TABLE_INDEX);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems also to me that OVERVIEW_TABLE_INDEX is at least not declared in the same class, so javac telling me it is missing this sounds likely.


int currentRow = 6;
int currentRow = 7;

UserService.Iface userClient = new ThriftClients().makeUserClient();

Expand All @@ -226,7 +241,12 @@ private void fillAttendeesTable(XWPFDocument document, Project project) throws X
continue;
}

User user = userClient.getByEmail(email);
User user = null;
try {
user = userClient.getByEmail(email);
} catch (TException te) {
// a resulting null user object is handled below by replacing with email
}

XWPFTableRow row = table.insertNewTableRow(currentRow++);
String name = email;
Expand All @@ -235,7 +255,7 @@ private void fillAttendeesTable(XWPFDocument document, Project project) throws X
}
String department = "N.A.";
if(user != null) {
name = user.getDepartment();
department = user.getDepartment();
}

row.addNewTableCell().setText(name);
Expand All @@ -247,7 +267,7 @@ private void fillAttendeesTable(XWPFDocument document, Project project) throws X
}

private void fillSpecialOSSRisksTable(XWPFDocument document, Project project, Collection<ObligationParsingResult> obligationResults) throws XmlException, TException {
XWPFTable table = document.getTables().get(1);
XWPFTable table = document.getTables().get(SPECIAL_OSS_RISKS_TABLE_INDEX);
final int[] currentRow = new int[]{0};

obligationResults.stream()
Expand All @@ -266,7 +286,7 @@ private void fillSpecialOSSRisksTable(XWPFDocument document, Project project, Co
}

private void fillOverview3rdPartyComponentTable(XWPFDocument document, Collection<LicenseInfoParsingResult> projectLicenseInfoResults) throws XmlException {
XWPFTable table = document.getTables().get(3);
XWPFTable table = document.getTables().get(THIRD_PARTY_COMPONENT_OVERVIEW_TABLE_INDEX);

int currentRow = 1;
for(LicenseInfoParsingResult result : projectLicenseInfoResults) {
Expand Down Expand Up @@ -298,17 +318,31 @@ private static Optional<ObligationParsingResult> obligationsForRelease(Release r
return obligationResults.stream().filter(opr -> opr.getRelease() == release).findFirst();
}

private void writeComponentSubsections(XWPFDocument document, Collection<LicenseInfoParsingResult> projectLicenseInfoResults, Collection<ObligationParsingResult> obligationResults) throws XmlException {
private void writeComponentSubsections(XWPFDocument document, Collection<LicenseInfoParsingResult> projectLicenseInfoResults, Collection<ObligationParsingResult> obligationResults) throws SW360Exception, XmlException {
XmlCursor cursor = document.getTables().get(ADDITIONAL_REQ_TABLE_INDEX).getCTTbl().newCursor();
cursor.toEndToken();

for (LicenseInfoParsingResult result : projectLicenseInfoResults) {
while (cursor.currentTokenType() != XmlCursor.TokenType.START && cursor.hasNextToken()) {
cursor.toNextToken();
}

XWPFParagraph title = document.createParagraph();
if (cursor.currentTokenType() != XmlCursor.TokenType.START) {
throw new SW360Exception("Corrupt template; unable find start token");
}

XWPFParagraph title = document.insertNewParagraph(cursor);
title.setStyle(STYLE_HEADING_3);
title.setNumID(new BigInteger("2"));
XWPFRun titleRun = title.createRun();
titleRun.setText(result.getVendor() + " " + result.getName());

XWPFParagraph description = document.createParagraph();
if (cursor.hasNextToken()) {
cursor.toNextToken();
} else {
throw new SW360Exception("Corrupt template; unable to proceed to next token");
}
XWPFParagraph description = document.insertNewParagraph(cursor);
XWPFRun descriptionRun = description.createRun();

LicenseInfo licenseInfo = result.getLicenseInfo();
Expand Down Expand Up @@ -337,7 +371,7 @@ private void writeComponentSubsections(XWPFDocument document, Collection<License

int currentRow = 0;
Collection<Obligation> obligations = obligationsResult.getObligations();
XWPFTable table = document.createTable();
XWPFTable table = document.insertNewTbl(cursor);
for (Obligation o : obligations) {
XWPFTableRow row = table.insertNewTableRow(currentRow++);
String licensesString = String.join(" ", o.getLicenseIDs());
Expand All @@ -349,29 +383,34 @@ private void writeComponentSubsections(XWPFDocument document, Collection<License
}
}

private void fillDevelopmentDetailsTable(XWPFDocument document, Project project, User user) throws TException {
private void fillDevelopmentDetailsTable(XWPFDocument document, Project project, User user, Collection<LicenseInfoParsingResult> projectLicenseInfoResults) throws TException {
XWPFTable table = document.getTables().get(DEV_DETAIL_TABLE_INDEX);

int currentRow = 1;

ComponentService.Iface compClient = new ThriftClients().makeComponentClient();
List<ReleaseLink> rll = compClient.getLinkedReleases(project.getReleaseIdToUsage());
for(LicenseInfoParsingResult result : projectLicenseInfoResults) {
if (result.getStatus() != LicenseInfoRequestStatus.SUCCESS) {
// this error handling is for extra safety since projectLicenseInfoResults is provided by the caller
// and we assume valid input so we silently ignoring it.
continue;
}

for (ReleaseLink rl : rll) {
Release r = compClient.getReleaseById(rl.getId(), user);
Component component = compClient.getComponentById(r.getComponentId(), user);
Release r = result.getRelease();
if (r == null) {
continue;
}

XWPFTableRow row = table.insertNewTableRow(currentRow++);

row.addNewTableCell().setText(component.getName());
row.addNewTableCell().setText(r.getName());

String operatingSystems = component.getOperatingSystemsSize() == 0 ? "Unknown operating systems" : String.join(" ", component.getOperatingSystems());
String operatingSystems = r.getOperatingSystemsSize() == 0 ? "N/A" : String.join(" ", r.getOperatingSystems());
row.addNewTableCell().setText(operatingSystems);

String langs = component.getLanguagesSize() == 0 ? "Unknown languages" : String.join(" ", component.getLanguages());
String langs = r.getLanguagesSize() == 0 ? "N/A" : String.join(" ", r.getLanguages());
row.addNewTableCell().setText(langs);

String platforms = component.getSoftwarePlatformsSize() == 0 ? "Unknown platforms" : String.join(" ", component.getSoftwarePlatforms());
String platforms = r.getSoftwarePlatformsSize() == 0 ? "N/A" : String.join(" ", r.getSoftwarePlatforms());
row.addNewTableCell().setText(platforms);
}
}
Expand All @@ -385,21 +424,19 @@ protected static Set<String> extractMostCommonLicenses(Collection<ObligationPars
.entrySet().stream()
.filter(entry -> entry.getValue().longValue() >= threshold)
.map(entry -> entry.getKey())
.map(license -> license.replace("\n", "").replace("\r", ""))
.collect(Collectors.toSet());
}

private void fillAdditionalRequirementsTable(XWPFDocument document, Collection<ObligationParsingResult> obligationResults) throws XmlException {
// extract licenses that appear at least ADDITIONAL_REQ_THRESHOLD times
Set<String> mostLicenses = extractMostCommonLicenses(obligationResults, ADDITIONAL_REQ_THRESHOLD);

private void fillAdditionalRequirementsTable(XWPFDocument document, Collection<ObligationParsingResult> obligationResults, Set<String> mostLicenses) throws XmlException {
XWPFTable table = document.getTables().get(ADDITIONAL_REQ_TABLE_INDEX);
final int[] currentRow = new int[]{0};

obligationResults.stream()
.filter(opr -> opr.getStatus() == ObligationInfoRequestStatus.SUCCESS)
.flatMap(opr -> opr.getObligations().stream())
.filter(o -> o.getLicenseIDs().stream()
.anyMatch(lid -> mostLicenses.parallelStream().anyMatch(mlid -> mlid.equals(lid))))
.anyMatch(lid -> mostLicenses.parallelStream().anyMatch(mlid -> mlid.equals(lid.replace("\n", "").replace("\r", "")))))
.forEach(o -> {
currentRow[0] = currentRow[0] + 1;
XWPFTableRow row = table.insertNewTableRow(currentRow[0]);
Expand Down
Binary file modified backend/src/src-licenseinfo/src/main/resources/templateReport.docx
View file
Open in desktop
Binary file not shown.
5 changes: 3 additions & 2 deletions ...w360-portlet/src/main/java/org/eclipse/sw360/portal/portlets/projects/ProjectPortlet.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -320,8 +320,9 @@ private void downloadLicenseInfo(ResourceRequest request, ResourceResponse respo
}

private void sendLicenseInfoResponse(ResourceRequest request, ResourceResponse response, Project project, LicenseInfoFile licenseInfoFile) throws IOException {
OutputFormatInfo outputFormatInfo = licenseInfoFile.getOutputFormatInfo();
String filename = String.format("LicenseInfo-%s%s-%s.%s", project.getName(),
OutputFormatInfo outputFormatInfo = licenseInfoFile.getOutputFormatInfo();
String documentVariant = licenseInfoFile.getOutputFormatInfo().getVariant() == OutputFormatVariant.DISCLOSURE ? "LicenseInfo" : "ProjectClearingReport";
String filename = String.format("%s-%s%s-%s.%s", documentVariant, project.getName(),
StringUtils.isBlank(project.getVersion()) ? "" : "-" + project.getVersion(),
SW360Utils.getCreatedOnTime().replaceAll("\\s", "_").replace(":", "_"),
outputFormatInfo.getFileExtension());
Expand Down
2 changes: 2 additions & 0 deletions frontend/sw360-portlet/src/main/webapp/html/components/editRelease.jsp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@

<core_rt:set var="programmingLanguages" value='<%=PortalConstants.PROGRAMMING_LANGUAGES%>'/>
<core_rt:set var="operatingSystemsAutoC" value='<%=PortalConstants.OPERATING_SYSTEMS%>'/>
<core_rt:set var="platformsAutoC" value='<%=PortalConstants.SOFTWARE_PLATFORMS%>'/>

<core_rt:set var="addMode" value="${empty release.id}"/>
<core_rt:set var="cotsMode" value="<%=component.componentType == ComponentType.COTS%>"/>
Expand Down Expand Up @@ -179,6 +180,7 @@
Liferay.on('allPortletsReady', function() {
autocomplete.prepareForMultipleHits('programminglanguages', ${programmingLanguages});
autocomplete.prepareForMultipleHits('op_systems', ${operatingSystemsAutoC});
autocomplete.prepareForMultipleHits('platformsTB', ${platformsAutoC});

sw360Validate.validateWithInvalidHandlerNoIgnore('#releaseEditForm');

Expand Down
28 changes: 18 additions & 10 deletions ...360-portlet/src/main/webapp/html/components/includes/releases/editReleaseInformation.jspf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
<table class="table info_table" id="ComponentBasicInfo">
<thead>
<tr>
<th colspan="3" class="headlabel">Release Summary</th>
<th colspan="4" class="headlabel">Release Summary</th>
</tr>
</thead>
<tr>
Expand Down Expand Up @@ -52,6 +52,17 @@
placeholder="e.g.,Linux,MAC,Windows,..."
value="<sw360:DisplayCollection value="${release.operatingSystems}" />"/>
</td>
<td width="33%">
<label class="textlabel stackedLabel" for="platformsTB">Software Platforms</label>
<input class="toplabelledInput" id="platformsTB"
name="<portlet:namespace/><%=Component._Fields.SOFTWARE_PLATFORMS%>" type="text" align="middle"
placeholder="e.g.,Adobe AIR,.NET,Qt,..."
value="<sw360:DisplayCollection value="${release.softwarePlatforms}" />"/>
</td>

</tr>

<tr>
<td width="33%">
<label class="textlabel stackedLabel" for="comp_id">CPE ID</label>
<input id="comp_id" name="<portlet:namespace/><%=Release._Fields.CPEID%>" type="text"
Expand All @@ -61,22 +72,13 @@
<img class="infopic" src="<%=request.getContextPath()%>/images/ic_info.png"
title="The formula for CPE ID creation is &#13;'cpe:2.3:a:VENDORNAME:COMPONENTNAME:VERSION' "/>
</td>
</tr>

<tr>
<td width="33%">
<label class="textlabel stackedLabel" for="releaseDate">Release Date</label>
<input id="releaseDate" class="datepicker" name="<portlet:namespace/><%=Release._Fields.RELEASE_DATE%>" type="text"
placeholder="Enter Release Date"
pattern="\d{4}-\d{2}-\d{2}"
value="<sw360:out value="${release.releaseDate}"/>"/>
</td>

<td width="33%">
<sw360:DisplayMainLicensesEdit id="<%=Release._Fields.MAIN_LICENSE_IDS.toString()%>"
licenseIds="${release.mainLicenseIds}"/>
</td>

<td width="33%">
<label class="textlabel stackedLabel" for="downloadUrl">Download URL</label>
<input id="downloadUrl" name="<portlet:namespace/><%=Release._Fields.DOWNLOADURL%>" type="URL"
Expand Down Expand Up @@ -133,6 +135,12 @@
description="Moderators" multiUsers="true" readonly="false"/>
</td>
</tr>
<tr>
<td width="33%">
<sw360:DisplayMainLicensesEdit id="<%=Release._Fields.MAIN_LICENSE_IDS.toString()%>"
licenseIds="${release.mainLicenseIds}"/>
</td>
</tr>
</table>

<script>
Expand Down
1 change: 1 addition & 0 deletions libraries/lib-datahandler/src/main/thrift/components.thrift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,7 @@ struct Release {
53: optional set<string> operatingSystems,
54: optional COTSDetails cotsDetails,
55: optional EccInformation eccInformation,
56: optional set<string> softwarePlatforms,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new field is completely independent from the list of software platforms at the component level? Why is it needed?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the first question. It is needed because we report on the release level and not on the component level.

Copy link
Contributor

@imaykay imaykay Jun 5, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already a field "softwarePlatforms" at the component level. As a release always belongs to a component I was wondering if the fields should really be independent. Or if there should be some logic like "the list of software platforms of a component is the aggregation of software platform lists of its releases" now that we have this field at release level.
Beside that your core idea here seems to be that the content of software platforms can change between different releases of the same component. Is that correct? Otherwise you could just get the information from the component of the release even when you report on release level.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is already a field "softwarePlatforms" at the component level. As a release always belongs to a component I was wondering if the fields should really be independent. Or if there should be some logic like "the list of software platforms of a component is the aggregation of software platform lists of its releases" now that we have this field at release level.

Since a release already has it's own "operating systems" & "programming languages" i assumed that "software platforms" should follow suit. If this is wrong, what is the purpose of "operating systems" & "programming languages" on the release level?

Beside that your core idea here seems to be that the content of software platforms can change between different releases of the same component. Is that correct? Otherwise you could just get the information from the component of the release even when you report on release level.

Yes, that is correct. Isn't the component suppose to be more of an abstract/template of a release and the each release is what actually goes into the project?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe the field softwarePlatforms should not been created at the first hand, but if in the report, it should be taken from the component where the release belongs to. I think a similar case is type (OSS, COTS, Freeware, ...) where the field value is taken from the component when listing releases.


65: optional set<string> mainLicenseIds,

Expand Down
5 changes: 4 additions & 1 deletion ...erver/src/main/java/org/eclipse/sw360/rest/resourceserver/core/JacksonCustomizations.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,10 @@ static abstract class ComponentMixin extends Component {
"contributorsIterator",
"rolesSize",
"setRoles",
"setCreatorDepartment"
"setCreatorDepartment",
"setSoftwarePlatforms",
"softwarePlatformsSize",
"softwarePlatformsIterator",
})
static abstract class ReleaseMixin extends Release {
@Override
Expand Down
Loading