Skip to content

Commit

Permalink
feat(analysis): add dialog option for large and -o for mimo (#3743)
Browse files Browse the repository at this point in the history
Also updates the MergeFilter README (copied into gh-pages), updates tests for dialog and adds test for output file during mimo
  • Loading branch information
phanlezz committed Dec 3, 2024
1 parent a1a4ff9 commit 86960be
Show file tree
Hide file tree
Showing 8 changed files with 228 additions and 62 deletions.
9 changes: 9 additions & 0 deletions analysis/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,17 @@ and this project adheres to [Semantic Versioning](http://semver.org/)

## [unreleased] (Added 🚀 | Changed | Removed 🗑 | Fixed 🐞 | Chore 👨‍💻 👩‍💻)

### Added 🚀

- Add a new `--large` flat to the MergeFilter that merges projects into one file each in its own subfolder depending on the input file's dot-prefix name [#3841](https://github.com/MaibornWolff/codecharta/pull/3841)
- Add the ability to the MergeFilter to specify the output file during `--mimo` operation [#3841](https://github.com/MaibornWolff/codecharta/pull/3841)

## [1.129.0] - 2024-11-29

### Added 🚀

- Add a new `--mimo` flag to the MergeFilter that merges multiple project files into multiple output files depending on the input file's dot-prefix name [#3800](https://github.com/MaibornWolff/codecharta/pull/3800)

### Changed

- Moved the structure print functionality to a new tool called 'inspection' [#3826](https://github.com/MaibornWolff/codecharta/pull/3826)
Expand Down
45 changes: 30 additions & 15 deletions analysis/filter/MergeFilter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,20 @@ Both strategies will merge the unique list entries for `attributeTypes` and `bla

## Usage and Parameters

| Parameters | Description |
|---------------------------------|----------------------------------------------------------------------|
| `FILE` | files to merge |
| `-a, --add-missing` | [Leaf Merging Strategy] enable adding missing nodes to reference |
| `-h, --help` | displays help and exits |
| `--ignore-case` | ignores case when checking node names |
| `--leaf` | use leaf merging strategy |
| `-nc, --not-compressed` | save uncompressed output File |
| `-o, --outputFile=<outputFile>` | output File (or empty for stdout; ignored in [MIMO mode])) |
| `--recursive` | use recursive merging strategy (default) |
| `--mimo` | merge multiple files with the same prefix into multiple output files |
| `-ld, --levenshtein-distance` | [MIMO mode] levenshtein distance for name match suggestions |
| `-f` | force merge non-overlapping modules at the top-level structure |
| Parameters | Description |
|---------------------------------|----------------------------------------------------------------------------------|
| `FILE` | files to merge |
| `-a, --add-missing` | [Leaf Merging Strategy] enable adding missing nodes to reference |
| `-h, --help` | displays help and exits |
| `--ignore-case` | ignores case when checking node names |
| `--leaf` | use leaf merging strategy |
| `-nc, --not-compressed` | save uncompressed output File |
| `-o, --outputFile=<outputFile>` | output File (or empty for stdout; [MIMO mode] output folder)) |
| `--recursive` | use recursive merging strategy (default) |
| `--mimo` | merge multiple files with the same prefix into multiple output files |
| `-ld, --levenshtein-distance` | [MIMO mode] levenshtein distance for name match suggestions |
| `-f` | force merge non-overlapping modules at the top-level structure |
| `--large` | merge multiple project files into one output file, separated by their dot-prefix |

```
Usage: ccsh merge [-ah] [--ignore-case] [--leaf] [-nc] [--recursive]
Expand All @@ -53,10 +54,24 @@ This last example inputs the folder foo, which will result in all project files
ccsh merge myProjectFolder/ --mimo -ld 0 -f
```

## MIMO - Multiply Inputs Multiple Outputs
## MIMO Merge - Multiply Inputs Multiple Outputs

Matches multiple `cc.json` files based on their prefix (e.g. **myProject**.git.cc.json). Tries to match project names with typos and asks which to add to the output.
If you want to use this in a CI/CD pipeline environment you may find it useful to specify `-ld` and `-f` to not prompt any user input.
The output file name follows the following schema: `myProject.merge.cc.json`.

> IMPORTANT: Output is always the current working directory.
## Large Merge

Merges multiple `.cc.json` files into one projects, but separates them into sub-folders, with names defined through the dot-prefixes of the input files:

```
ccsh merge aa.cc.json bb.cc.json cc.cc.json --large -o myOutputFile -nc
# myOutputFile.cc.json:
# - root
# - - aa
# - - - *
# - - bb
# - - - *
# - - cc
# - - - *
```
Original file line number Diff line number Diff line change
Expand Up @@ -175,10 +175,12 @@ class MergeFilter(

val mergedProject = ProjectMerger(projects, nodeMergerStrategy).merge()
val outputFilePrefix = Mimo.retrieveGroupName(projectsFileNamePairs.map { it.first })
ProjectSerializer.serializeToFileOrStream(mergedProject, "$outputFilePrefix.merge.cc.json", output, compress)
val outputFileName = "$outputFilePrefix.merge.cc.json"
val outputFilePath = Mimo.assembleOutputFilePath(outputFile, outputFileName)
ProjectSerializer.serializeToFileOrStream(mergedProject, outputFilePath, output, compress)
Logger.info {
"Merged files with prefix '$outputFilePrefix' into" +
" '$outputFilePrefix.merge.cc.json${if (compress) ".gz" else ""}'"
" '$outputFileName${if (compress) ".gz" else ""}'"
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,24 +20,32 @@ class ParserDialog {
inputFolderName = getInputFileName("cc.json", InputType.FOLDER)
} while (!InputHelper.isInputValidAndNotNull(arrayOf(File(inputFolderName)), canInputContainFolders = true))

val isMimoMode = KInquirer.promptConfirm(
message = "Do you want to use MIMO mode? (multiple inputs multiple outputs)",
default = false
val defaultMerge = "Default merging..."
val mimoMerge = "Mimo Merge"
val largeMerge = "Large Merge"
val mergeMode = KInquirer.promptList(
message = "Do you want to use a special merge mode?",
choices = listOf(defaultMerge, mimoMerge, largeMerge)
)

var outputFileName = ""
val outputFileName: String
val isCompressed: Boolean
var levenshteinDistance = 0
if (isMimoMode) {
levenshteinDistance = KInquirer.promptInputNumber(
message = "Select Levenshtein Distance for name match suggestions (0 for no suggestions)",
default = "3"
).toInt()
if (mergeMode == mimoMerge) {
outputFileName = KInquirer.promptInput(
message = "What is the output folder path?",
hint = "Uses the current working directory if empty"
)

isCompressed = KInquirer.promptConfirm(
message = "Do you want to compress the output file(s)?",
default = true
)

levenshteinDistance = KInquirer.promptInputNumber(
message = "Select Levenshtein Distance for name match suggestions (0 for no suggestions)",
default = "3"
).toInt()
} else {
outputFileName =
KInquirer.promptInput(
Expand Down Expand Up @@ -82,18 +90,24 @@ class ParserDialog {
"--recursive=${!leafFlag}",
"--leaf=$leafFlag",
"--ignore-case=$ignoreCase",
"--not-compressed=$isCompressed"
"--not-compressed=$isCompressed",
"--output-file=$outputFileName"
)

if (isMimoMode) {
if (mergeMode == mimoMerge) {
return basicMergeConfig + listOf(
"--mimo=true",
"--levenshtein-distance=$levenshteinDistance"
)
}
return basicMergeConfig + listOf(
"--output-file=$outputFileName"
)

if (mergeMode == largeMerge) {
return basicMergeConfig + listOf(
"--large=true"
)
}

return basicMergeConfig
}

fun askForceMerge(): Boolean {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,5 +83,15 @@ class Mimo {

return cost[rhsLength]
}

fun assembleOutputFilePath(filePath: String?, fileName: String): String {
return if (filePath.isNullOrEmpty()) {
fileName
} else if (File(filePath).isDirectory) {
"${File(filePath).path}/$fileName"
} else {
throw IllegalArgumentException("Please specify a folder for MIMO output or nothing")
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -493,11 +493,54 @@ class MergeFilterTest {

outputFile.deleteOnExit()
}

@Test
fun `should merge two projects with output at the given location`() {
val prefix = "testProject"
val outputFolder = "src/test/resources"

System.setErr(PrintStream(errContent))
CommandLine(MergeFilter()).execute(
testProjectPathA,
testProjectPathB,
"--mimo",
"--levenshtein-distance=0",
"-o=$outputFolder"
).toString()

System.setErr(originalErr)

val outputFileName = "$prefix.merge.cc.json.gz"
val outputFile = File("$outputFolder/$outputFileName")

assertThat(errContent.toString()).contains("Merged files with prefix '$prefix' into '$outputFileName'")
assertThat(outputFile).exists()

outputFile.deleteOnExit()
}

@Test
fun `should throw error if output-file is not a folder`() {
val invalidOutputPath = "src/test/resources/invalid.cc.json"

System.setErr(PrintStream(errContent))
CommandLine(MergeFilter()).execute(
testProjectPathA,
testProjectPathB,
"--mimo",
"--levenshtein-distance=0",
"-o=$invalidOutputPath"
).toString()

System.setErr(originalErr)

assertThat(errContent.toString()).contains("Please specify a folder for MIMO output or nothing")
}
}

@Nested
@DisplayName("FatMergeTests")
inner class FatMergeTest {
@DisplayName("LargeMergeTests")
inner class LargeMergeTest {
private val fatMergeTestFolder = "src/test/resources/largeMerge"
private val testFilePath1 = "$fatMergeTestFolder/testEdges1.cc.json"
private val testFilePath2 = "$fatMergeTestFolder/testProject.alpha.cc.json"
Expand Down
Loading

0 comments on commit 86960be

Please sign in to comment.