Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add nextVersion and untagged deprecations #82

Merged
merged 1 commit into from
Aug 29, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,5 +19,6 @@
"outputDirectory": "./deprecations",
"tsConfigPath": "tsconfig.sandbox.json",
"deprecationComment": "@deprecated",
"deprecationLink": "sandbox-deprecation-link"
}
"deprecationLink": "sandbox-deprecation-link",
"commentLinkFormat": "Details: {@link ${COMMENT_LINK_URL_TOKEN}#${COMMENT_LINK_URL_PARAM_TOKEN}}"
}
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<!-- ruid-groups

- master:
- /tree/master/all-lowercase/crawled.ts#L12
- /tree/master/all-lowercase/crawled.ts#L18
- /tree/master/all-lowercase/crawled.ts#L24
- :
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just see this, this probably has to be fixed 🤔
But this also raises the question if untagged versions should be included in the markdown output files.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice pr! I'll have a think on that!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about giving it the "version" of the main branch until there the next tag arrives?

This most probably introduces more fiddly logic, so I'm not sure if this is a good idea,.
But at least we have the conversation about the problem ;)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's maybe too complex for now 😅

The current implementation of the group markdowns is that it will append new deorecations to the file, correct (meaning that you could make manual edits to the existing deprecations in the file, and it won't get overwritten)?

I also have a second question, does it make sense to have the checkout step?
I mean, if you go to a different branch, and you pass in a next version, that seems a little bit odd to me.
In my opinion (I could be wrong), we don't need the checkout step when we run the crawler in its normal mode. It will just crawl the current branch.
If we implement the history command, that's the command where we should use the checkout step (possibly multiple times).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation of the group markdowns is that it will append new deprecations to the file, correct (meaning that you could make manual edits to the existing deprecations in the file, and it won't get overwritten)?

It went to the top of the file and replaces the whole comment block with all the deprecations form raw-depreciations.
So whatever is done in the file stays untouched, but the comments are completely new. All edits in the comments are lost which is no problem as comments.. :D

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also have a second question, does it make sense to have the checkout step?

No, but the last valid tag should get determined right?

How does it work in master? I can imagine in the output formatted we can replace empty versions with the main name?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, you've merged this one in.
Just as an fyi, I've taken a closer look to the generated markdown and I agree with you 😁
I will create a follow up PR to add the main branch 😉

- /tree//all-lowercase/crawled.ts#L12
- /tree//all-lowercase/crawled.ts#L18
- /tree//all-lowercase/crawled.ts#L24

ruid-groups -->

Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<!-- ruid-groups

- master:
- /tree/master/multiple-string-patterns-at-once/crawled.ts#L15
- /tree/master/multiple-string-patterns-at-once/crawled.ts#L21
- /tree/master/multiple-string-patterns-at-once/crawled.ts#L27
- :
- /tree//multiple-string-patterns-at-once/crawled.ts#L15
- /tree//multiple-string-patterns-at-once/crawled.ts#L21
- /tree//multiple-string-patterns-at-once/crawled.ts#L27

ruid-groups -->

Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
<!-- ruid-groups

- master:
- /tree/master/deprecation-comments/crawled.ts#L6
- /tree/master/deprecation-comments/crawled.ts#L14
- /tree/master/deprecation-comments/crawled.ts#L24
- /tree/master/deprecation-comments/crawled.ts#L29
- /tree/master/deprecation-comments/crawled.ts#L34
- :
- /tree//deprecation-comments/crawled.ts#L6
- /tree//deprecation-comments/crawled.ts#L14
- /tree//deprecation-comments/crawled.ts#L24
- /tree//deprecation-comments/crawled.ts#L29
- /tree//deprecation-comments/crawled.ts#L34

ruid-groups -->

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
173,
311
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "3998890672",
Expand All @@ -27,7 +27,7 @@
338,
417
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "2129363187",
Expand All @@ -44,7 +44,7 @@
444,
528
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "2585813938",
Expand All @@ -61,7 +61,7 @@
73,
129
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "3397214801",
Expand All @@ -78,7 +78,7 @@
171,
255
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "456802929",
Expand All @@ -95,7 +95,7 @@
299,
433
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "242198417",
Expand All @@ -112,7 +112,7 @@
477,
557
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "2601492017",
Expand All @@ -129,7 +129,7 @@
601,
649
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "2711487569",
Expand All @@ -146,7 +146,7 @@
240,
459
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "1734398741",
Expand All @@ -163,7 +163,7 @@
494,
638
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "3477709142",
Expand All @@ -180,7 +180,7 @@
673,
823
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "2417065367",
Expand All @@ -197,7 +197,7 @@
146,
306
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "3186155811",
Expand All @@ -214,7 +214,7 @@
348,
472
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "677771456",
Expand All @@ -231,7 +231,7 @@
514,
618
],
"version": "master",
"version": "5.4.2",
"remoteUrl": "",
"date": "",
"ruid": "3130163297",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
<!-- ruid-groups

- master:
- /tree/master/whitespace-normalisation/crawled.ts#L12
- /tree/master/whitespace-normalisation/crawled.ts#L18
- /tree/master/whitespace-normalisation/crawled.ts#L24
- :
- /tree//whitespace-normalisation/crawled.ts#L12
- /tree//whitespace-normalisation/crawled.ts#L18
- /tree//whitespace-normalisation/crawled.ts#L24

ruid-groups -->

Expand Down
3 changes: 1 addition & 2 deletions packages/deprecation-crawler/src/lib/crawler/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,7 @@ function crawlFileForDeprecations(
deprecation.comment.range.compilerObject.pos,
deprecation.comment.range.compilerObject.end,
],
// @TODO consider storing the tag directly
version: crawledRelease.tag,
version: '',
// @TODO consider moving it tothe formatting step and keep the deprecation obj small.
remoteUrl: crawledRelease.remoteUrl,
date: crawledRelease.date,
Expand Down
8 changes: 6 additions & 2 deletions packages/deprecation-crawler/src/lib/deprecation-crawler.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { getConfig } from './config';
import { CrawledRelease } from './models';
import { stripIndent } from 'common-tags';
import { branchHasChanges, run } from './utils';
import { branchHasChanges, getVersion, run } from './utils';
import { logError } from './log';
import { checkout } from './tasks/checkout';
import { crawl } from './tasks/crawl';
Expand All @@ -10,6 +10,7 @@ import { addGroups } from './tasks/add-groups';
import { generateOutput } from './tasks/generate-output';
import { commitChanges } from './tasks/commit-changes';
import { CRAWLER_MODES } from './constants';
import { addVersion } from './tasks/add-version';

(async () => {
await guardAgainstDirtyRepo();
Expand All @@ -19,14 +20,17 @@ import { CRAWLER_MODES } from './constants';
const tasks = [
checkout,
crawl,
addVersion,
addGroups,
generateOutput,
updateRepository,
commitChanges,
];

// Run all processors
const initial = ({} as unknown) as CrawledRelease;
const initial = {
version: getVersion(),
} as CrawledRelease;
run(tasks, config)(initial);
})();

Expand Down
1 change: 1 addition & 0 deletions packages/deprecation-crawler/src/lib/models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ export interface RawDeprecation {

export interface CrawledRelease {
tag: string;
version: string;
date: string;
remoteUrl: string;
deprecations: Deprecation[];
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
import { readFileSync, writeFileSync } from 'fs';
import { join } from 'path';
import { CrawlConfig, CrawledRelease, Deprecation } from '../models';
import { ensureDirExists } from '../utils';
import { RAW_DEPRECATION_PATH } from '../constants';
import {
ensureDirExists,
readRawDeprecations,
writeRawDeprecations,
} from '../utils';
import * as kleur from 'kleur';

export async function generateRawJson(
Expand All @@ -11,23 +12,16 @@ export async function generateRawJson(
): Promise<void> {
ensureDirExists(config.outputDirectory);

let existingDeprecations: Deprecation[] = [];
try {
const t = readFileSync(
join(config.outputDirectory, `${RAW_DEPRECATION_PATH}`)
);
existingDeprecations = JSON.parse((t as unknown) as string);
} catch (e) {
existingDeprecations = [];
}
const { deprecations: existingDeprecations, path } = readRawDeprecations(
config
);

const deprecations = upsertDeprecations(
existingDeprecations,
crawledRelease.deprecations
);

const json = JSON.stringify(deprecations, null, 4);
const path = join(config.outputDirectory, `${RAW_DEPRECATION_PATH}`);
writeFileSync(path, json);
writeRawDeprecations(deprecations, config);

console.log(kleur.gray(`📝 Raw JSON data up to date under ${path}`));
}
Expand Down
46 changes: 46 additions & 0 deletions packages/deprecation-crawler/src/lib/tasks/add-version.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import {
CrawlConfig,
CrawledRelease,
CrawlerProcess,
Deprecation,
} from '../models';
import {
concat,
readRawDeprecations,
tap,
writeRawDeprecations,
} from '../utils';

/**
* Adds the version to the (existing and new) deprecations
*/
export function addVersion(config: CrawlConfig): CrawlerProcess {
return async function (release) {
if (!release.version) return release;

return concat([
async (r): Promise<CrawledRelease> => {
return {
...r,
deprecations: updateVersion(r.deprecations, r.version),
};
},
tap(async (r) => updateExistingDeprecations(config, r.version)),
])(release);
};
}

function updateVersion(
rawDeprecations: Deprecation[],
version: string
): Deprecation[] {
return rawDeprecations.map((deprecation) => {
return { ...deprecation, version: version };
});
}

function updateExistingDeprecations(config: CrawlConfig, version: string) {
const { deprecations } = readRawDeprecations(config);
const deprecationsWithVersion = updateVersion(deprecations, version);
writeRawDeprecations(deprecationsWithVersion, config);
}
Loading