Skip to content

Commit 29934c1

Browse files
orlikoskicbrycebbchapinbtobraha
authored
CyLR 2.2
* Addition of collection features. See CHANGELOG.md * Travis build fix * Travis build fix * Travis build fix * Travis build fix * Travis build fix * Travis build fix * Travis build fix * Updating to DotNet3.1 * Updating to DotNet3.1 * Updating to include single file output * Updating to include single file output * Updating to include single file output * Updating travis build file to work with new x86 and x64 windows binaries * Updating travis build files for macos/linux to work * Updating to allow for single file builds on all OS types * Updating to allow for single file builds on all OS types * Updating to use bionic for build system in travis-ci * Merged branch changes * Update package.sh Fixing merges * Removed timestamp from logging tests * Hide warn, and lower priority, messages from tests This only impacts collection logs and is meant to supress warnings about paths that are not accessible by the current account. * Added missing zip command definition Removed unused warp packer * Removing tests for Linux and MacOS as they're broken and the tests need to be updated to account for building in travis-ci. * fixing package.sh file header * Make package.sh build.sh test.sh executeable * Fixing publish script * enabling linux tests * add Chromium-based Edge History file Signed-off-by: Tommy Harris <[email protected]> Co-authored-by: Chapin Bryce <[email protected]> Co-authored-by: Chapin Bryce <[email protected]> Co-authored-by: Tommy Harris <[email protected]>
1 parent 03ed2be commit 29934c1

23 files changed

+2231
-342
lines changed

.gitignore

+4
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
CyLR/bin/
22
CyLR/obj/
3+
.DS_Store
34
.vs/
5+
.vscode/
46
packages/
57
*.vspx
68
*.db
@@ -13,3 +15,5 @@ CyLRTests/bin/
1315
*.zip
1416
deployments/
1517
warp-packer
18+
lcov.info
19+
*.log

.travis.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ script:
2929
- if [[ "$TRAVIS_OS_NAME" == "windows" ]]; then PowerShell -File scripts/build_win.ps1 ; fi
3030
- if [[ "$TRAVIS_OS_NAME" == "windows" ]]; then PowerShell -File scripts/package_win.ps1 ; fi
3131

32-
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then BUILD_ARCH=osx-x64 USE_CORERT=TRUE scripts/test.sh ; fi
32+
#- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then BUILD_ARCH=osx-x64 USE_CORERT=TRUE scripts/test.sh ; fi
3333
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then BUILD_ARCH=osx-x64 USE_CORERT=TRUE scripts/build.sh ; fi
3434
- if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then BUILD_ARCH=osx-x64 USE_CORERT=TRUE scripts/package.sh ; fi
3535

CHANGELOG.md

+60
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# CHANGE LOG
2+
3+
## 2.2.0 - 2020-07-09
4+
5+
This version includes numerous modifications and introduction of new features,
6+
highlighted below:
7+
8+
### Added
9+
10+
* Logging is available, to destinations including the console, a log file, and
11+
embedded within the resulting archive. The log name is specified with `-l`
12+
and verbosity is adjusted with `-v` to increase or `-q` to silence.
13+
* Added `CUSTOM_PATH_TEMPLATE.txt` with documentation on how to specify custom
14+
paths for collection.
15+
* Implemented enumeration of files system contents in the same manner cross
16+
platform
17+
* Through new FS enumeration, eliminated extra scanning/duplicate collection
18+
of data within symbolic link directories. Eliminated dependency on the
19+
`find` binary.
20+
* Enabled the use of globbing patterns within paths. This includes patterns
21+
such as:
22+
* `**/*.plist`
23+
* `/home/*/.*sh_history`
24+
* `\Windows\Temp\[a-z0-9][a-z0-9][a-z0-9][a-z0-9]\*`
25+
* `**/Library/*Support/Google/Chrome/Default/History*`
26+
* Enabled the use of regular expressions within paths. This includes full line
27+
and substring patterns, such as:
28+
* `.*mawlare.*`
29+
* `^C:\Windows\Temp\[A-Za-z0-9]{8}\.*$`
30+
* `^C:\Windows\System32\Config\(SOFTWARE|SYSTEM|SAM|SECURITY).*$`
31+
* Added functionality to allow the user to select whether the existence of a
32+
custom collection list (`-c`) should be in addition to versus in place of
33+
the default artifact list. Continues to default to the replacement option
34+
where it will only collect specified files.
35+
* Modified config file to support specification of path pattern type. Can be
36+
one of `static`, `glob`, or `regex`. Format should be a tab delimited text
37+
file with one pattern type and path per line. A line starting with a pound
38+
character will be ignored.
39+
* Provided status messages to summarize the number of files scanned and paths
40+
staged for collection.
41+
* Increased documentation of source code.
42+
43+
### Removed
44+
45+
* Removed collection of Windows Search path due to large size on some systems
46+
(`%PROGRAMDATA%\Microsoft\Search\Data\Applications\Windows`).
47+
Please use `-c` to re-include as needed.
48+
49+
### Changed
50+
51+
* Edited build scripts to point to C:\Program Files\7z instead of x86 folder
52+
* Improved the default collection paths for Linux platforms.
53+
* Modified the USNJrnl collection argument to default to disabled collection.
54+
* Improved SFTP handling to collect to a local zip file and attempt the
55+
upload three times, with a 30 second delay between attempts.
56+
* Semantic changes to packaging and build scripts to avoid alias use.
57+
* Added packaging script check to see if packaging tool was local before
58+
re-downloading.
59+
* Updated argument usage information.
60+
* Added tests to increase coverage of Arguments.cs

CUSTOM_PATH_TEMPLATE.txt

+177
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
#===========================================#
2+
# CyLR Custom File Path Collection Template #
3+
#===========================================#
4+
#
5+
# This file contains example methods for specifying file or folder paths to
6+
# collect with CyLR.
7+
#
8+
9+
#################
10+
# General Usage #
11+
#################
12+
#
13+
# CyLR contains default paths per-operating system that it will collect if you
14+
# do not specify this file. This file allows you to replace or extend the
15+
# default collection capability of CyLR.
16+
#
17+
# To configure CyLR to only collect the paths specified in this file, you can
18+
# provide this file at execution time as `CyLR.exe -c custom.txt` (where
19+
# custom.txt is this file's name and relative location). If you would like to
20+
# collect both default and custom items, you can accomplish this with `-d` in
21+
# place of `-c`. If you only want to collect default items, you can omit either
22+
# argument.
23+
#
24+
25+
###################################
26+
# Specifying custom path patterns #
27+
###################################
28+
#
29+
# This configuration file supports comments, meaning lines with a leading `#`
30+
# are ignored and not evaluated further; this is for the purpose of custom file
31+
# documentation, such as in this file, and for your own documentation as you
32+
# build out custom configurations for collections.
33+
#
34+
# When specifying a file or folder path to collect, you will want to use the
35+
# format of:
36+
# {pattern_type} {file_or_folder_path}
37+
#
38+
# This format specifies the pattern type and target folder or file path,
39+
# delimited by a tab character. *Please be sure to validate that the delimiter
40+
# is a tab character and not a series of space characters.*
41+
#
42+
# *Please note distinction between file and folder pattern support.*
43+
#
44+
# There are four supported pattern types:
45+
# * static -> Full path specification to a specific file
46+
# * glob -> Supports glob patterns for collection, such as `/path/**` or
47+
# `/Users/*/.*history`. Where possible, this should be used in place
48+
# of regex due to performance. Most common use case is to recursively
49+
# collect a path with a pattern such as `/path/**`.
50+
# * regex -> Support regular expressions in paths.
51+
# * force -> Try to collect a static path even if the path is not found during
52+
# the file system enumeration.
53+
#
54+
# There is no requirements for the file path specified so long as it is tab
55+
# delimited from the pattern type. The file path can contain spaces and special
56+
# characters so long as the dotnet file system library is able to interpret it.
57+
#
58+
# Please note that in order to collect all items in a folder, or recursively on
59+
# a path, you will need to use glob paths. We will dive into more details for
60+
# each of these pattern types below.
61+
#
62+
63+
#***************************#
64+
# * IMPORTANT INFORMATION * #
65+
#***************************#
66+
#
67+
# All patterns (except force) will attempt to match against an enumeration of
68+
# files on a system. This enumeration is performed by an internal function and
69+
# is subject to the permissions granted at runtime. If a discovered path is
70+
# responsive to a pattern, it is added to the collection queue.
71+
#
72+
# The `force` pattern type specifies to attempt collection regardless of
73+
# whether the file is available through enumeration. For this reason, it must
74+
# be a static full path for collection.
75+
#
76+
# All pattern matching is case insensitive.
77+
#
78+
# Please specify the path delimiter for the system targeted. This means "/" for
79+
# UNIX-like systems and "\" for Windows systems.
80+
#
81+
82+
#+++++++++++++++++#
83+
# Static patterns #
84+
#+++++++++++++++++#
85+
#
86+
# This pattern format is the most basic and can be used to collect a single
87+
# file at a known location. This path must be discovered during file system
88+
# enumeration to queue for collection.
89+
#
90+
# Examples:
91+
static C:\Windows\System32\Config\SOFTWARE
92+
static /etc/group
93+
94+
#++++++++++++++++#
95+
# Force patterns #
96+
#++++++++++++++++#
97+
#
98+
# This pattern format is the the same as static, but will add the file to the
99+
# collection queue regardless of whether the file is present in the file system
100+
# enumeration operation.
101+
#
102+
# This is useful for collecting hidden/hard to access artifacts such as
103+
# alternate data streams.
104+
#
105+
# Examples:
106+
force C:\$Extend\$UsnJrnl:$J
107+
108+
109+
#++++++++++++++++#
110+
# Glob patterns #
111+
#++++++++++++++++#
112+
#
113+
# This pattern format is useful for when a basic pattern is needed to collect
114+
# responsive files.
115+
#
116+
# Recursive collections
117+
#
118+
# The most common use case is for collecting files recursively. This is
119+
# achieved through using two sequential asterisk characters at the tail end of
120+
# the path.
121+
#
122+
# Examples:
123+
glob /var/log/**
124+
125+
# Wildcards
126+
#
127+
# Another common use case is the wildcard, specified as a single asterisk
128+
# character within a path segment. This Wildcard will match any character
129+
# except a path delimiter. This means we can specify patterns that match a
130+
# particular file name within any sub directory.
131+
#
132+
# We can also use wildcards to match on parts of a file name within a path
133+
# segment. Some use cases include collecting all files with a particular
134+
# extension or all files with the same name but differing extensions.
135+
#
136+
# Examples:
137+
glob C:\Users\*\NTUser.dat
138+
glob /home/*/.bash_history
139+
glob C:\Users\*\Appdata\Roaming\Microsoft\Windows\Recent\*.lnk
140+
glob /root/.*_history
141+
142+
# Other patterns
143+
#
144+
# Globbing supports more patterns specified here:
145+
# * https://github.com/dazinator/DotNet.Glob/blob/a21476b1078a3b49070bdb4bfde07030739e641d/README.md
146+
#
147+
# You can also combine the patterns into a single line, allowing you to specify
148+
# patterns like those below. While regex may provide a more succinct manner
149+
# to specify the pattern, glob should be favored for performance reasons.
150+
#
151+
# Examples:
152+
glob C:\Windows\Temp\[A-Z][A-Z][A-Z][A-Z]\*.exe
153+
glob C:\ProgramData\[A-Z][A-Z][A-Z][A-Z].exe
154+
glob C:\**\*.docm
155+
glob C:\**\r*.exe
156+
157+
#++++++++++++++++#
158+
# Regex patterns #
159+
#++++++++++++++++#
160+
#
161+
# Allows the specification of advanced patterns for detection. This is the
162+
# slowest method and should be leveraged if pattern functionality available
163+
# through the other options is insufficient.
164+
#
165+
# This supports patterns specified by dotnet's regex implementation, documented
166+
# here:
167+
#
168+
#
169+
# Please note that the patterns are matched as a substring across a full path.
170+
# This means a pattern of `[0-9]{8}` would match on any file system path
171+
# containing eight numbers sequentially. You can leverage `^` to indicate the
172+
# start of a file path, and `$` to denote the end.
173+
#
174+
# Examples:
175+
regex ^C:\ProgramData\[A-Za-z]{8}\[A-Za-z]{5}.exe$
176+
regex [0-9]+.exe
177+

CyLR/CyLR.csproj

+3-2
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,18 @@
1111
<PublishSingleFile>true</PublishSingleFile>
1212
<PublishReadyToRunShowWarnings>false</PublishReadyToRunShowWarnings>
1313
<Win32Resource />
14-
<Version>2.2.0</Version>
14+
<Version>3.0.0</Version>
1515
<RepositoryUrl>https://github.com/orlikoski/CyLR</RepositoryUrl>
1616
<PackageLicenseUrl>https://github.com/orlikoski/CyLR/blob/master/LICENSE</PackageLicenseUrl>
1717
<Copyright>2018</Copyright>
1818
<PackageIconUrl>Martin-Berube-Character-Knight.ico</PackageIconUrl>
1919
<TargetLatestRuntimePatch>true</TargetLatestRuntimePatch>
20+
<CopyOutputSymbolsToPublishDirectory>false</CopyOutputSymbolsToPublishDirectory>
2021
</PropertyGroup>
2122

2223
<ItemGroup>
2324
<PackageReference Include="Discutils" Version="0.13.0-alpha" />
24-
<!--<PackageReference Include="Microsoft.DotNet.ILCompiler" Version="1.0.0-alpha-*" Condition="'$(USE_CORERT)' == 'TRUE'" />-->
25+
<PackageReference Include="DotNet.Glob" Version="3.1.0-alpha0009" />
2526
<PackageReference Include="RawDiskLib" Version="0.1.4" />
2627
<PackageReference Include="SharpZipLib" Version="1.0.0" />
2728
<PackageReference Include="SSH.NET" Version="2016.1.0" />

0 commit comments

Comments
 (0)