Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: no adapter classes #1382

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
63ce40e
fix(perf): fewer adapter classes
mshanemc Jun 25, 2024
fd40840
refactor: derive metadataWithContent directly from type, not type => …
mshanemc Jun 25, 2024
70cf34c
test: remove invalid test case
mshanemc Jun 25, 2024
3095134
chore: bump core for xnuts
mshanemc Jun 25, 2024
08371d0
wip: clean up existing classes
mshanemc Jun 25, 2024
5b6d0ac
wip: even less classy adapters
mshanemc Jun 27, 2024
2ebc8de
Merge remote-tracking branch 'origin/main' into sm/no-adapter-classes
mshanemc Jun 30, 2024
f69a337
docs: typo
mshanemc Jul 15, 2024
e4414d0
refactor: no adapter classes
mshanemc Jul 15, 2024
018257d
Merge remote-tracking branch 'origin/main' into sm/no-adapter-classes
mshanemc Jul 26, 2024
496a493
refactor: organize adapter shared types
mshanemc Jul 26, 2024
d542496
chore: rename deb bundle detector
mshanemc Jul 30, 2024
3b93cc4
fix: get deb to resolve properly
mshanemc Jul 31, 2024
61b779d
chore: stash wip
mshanemc Jul 31, 2024
e0b80bb
test: set the contents for a forceIgnore. handy for ut
mshanemc Aug 1, 2024
5a0ff3c
test: passing ut and snapshot
mshanemc Aug 1, 2024
1727cd3
Merge remote-tracking branch 'origin/main' into sm/no-adapter-classes
mshanemc Aug 1, 2024
fffaea9
fix: allow more undefined from adapters
mshanemc Aug 1, 2024
42eb6d7
refactor: shared posixify fn for windows ut
mshanemc Aug 1, 2024
6e91e38
docs: handbook update
mshanemc Aug 1, 2024
0f149cc
docs: remove wip readme for adapters
mshanemc Aug 1, 2024
78ba78c
test: windows ut need posix paths in forceignore contents
mshanemc Aug 2, 2024
1ff3f66
test: ut for forceignore injection
mshanemc Aug 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,996 changes: 409 additions & 1,587 deletions CHANGELOG.md

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions HANDBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,7 @@ The resolver constructs components based on the rules of such a pattern. It take

1. Determine the associated type by parsing the file suffix. Utilize the registry indexes to determine a type
2. If the type has a source adapter assigned to it, construct the associated adapter. Otherwise use the default one
3. Call the adapter’s `getComponent()` method to construct the source component
3. Call the adapter function to construct the source component

📝 _CAREFULLY_ _consider whether new adapters need to be added. Ideally, we should never have to add another one and new types should follow existing conventions to reduce maintenance burden._

Expand Down Expand Up @@ -263,7 +263,7 @@ We'll continue to see examples of this as we look at the `strategies` that can b
These strategies are optional, but all have a default transformer, and converter assigned to them, which assumes nothing special needs to be done and that their source and metadata format are identical, _link to these files_.
Luckily, lots of type use the default transformers and adapters.

The `strategies` property, of a metadatata type entry in the registry, can define four properties, the `adapter`,`transformer`,`decompositon`, and `recomposition`. How SDR uses these values is explained more in detail later on, but we'll go through each of the options for these values, what they do, what behavior they enable, and the types that use them.
The `strategies` property, of a metadata type entry in the registry, can define four properties, the `adapter`,`transformer`,`decomposition`, and `recomposition`. How SDR uses these values is explained more in detail later on, but we'll go through each of the options for these values, what they do, what behavior they enable, and the types that use them.

The "adapters", or "source adapters", are responsible for understanding how a metadata type should be represented in source format and recognizing that pattern when constructing `Component Sets`

Expand Down Expand Up @@ -293,8 +293,8 @@ layouts/

### The `bundleSourceAdapter`

Like the name suggests, this adapter handles bundle types, so `AuraDefinitionBundles`, `LightningWebComponents`. A bundle component has all its source files, including the root metadata xml, contained in its own directory.

Like the name suggests, this adapter handles bundle types, so `AuraDefinitionBundles`, `LightningWebComponents`. A bundle component has all its source files contained in its own directory.
Most bundle types have a root metadata xml, but it's not required (ex: WaveTemplateBundle) and it might not be an xml file (ExperiencePropertyTypeBundle)
**Example Structure**:

```text
Expand Down
4 changes: 3 additions & 1 deletion src/client/deployMessages.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import { basename, dirname, extname, join, posix, sep } from 'node:path/posix';
import { SfError } from '@salesforce/core';
import { ensureArray } from '@salesforce/kit';
import { posixify } from '../utils/path';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a single function to do this

import { ComponentLike, SourceComponent } from '../resolve';
import { registry } from '../registry/registry';
import {
Expand Down Expand Up @@ -154,8 +155,9 @@ const hasComponentType = (message: DeployMessage): message is DeployMessage & {

export const toKey = (component: ComponentLike): string => {
const type = typeof component.type === 'string' ? component.type : component.type.name;
return `${type}#${shouldConvertPaths ? component.fullName.split(sep).join(posix.sep) : component.fullName}`;
return `${type}#${shouldConvertPaths ? posixify(component.fullName) : component.fullName}`;
};

const isTrue = (value: BooleanString): boolean => value === 'true' || value === true;

export const shouldConvertPaths = sep !== posix.sep;
4 changes: 2 additions & 2 deletions src/client/metadataApiDeploy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import JSZip from 'jszip';
import fs from 'graceful-fs';
import { Lifecycle, Messages, SfError } from '@salesforce/core';
import { ensureArray } from '@salesforce/kit';
import { posixify } from '../utils/path';
import { RegistryAccess } from '../registry/registryAccess';
import { ReplacementEvent } from '../convert/types';
import { MetadataConverter } from '../convert/metadataConverter';
Expand Down Expand Up @@ -311,8 +312,7 @@ export class MetadataApiDeploy extends MetadataTransfer<
// Add relative file paths to a root of "zip" for MDAPI.
const relPath = join('zip', relative(mdapiPath, fullPath));
// Ensure only posix paths are added to zip files
const relPosixPath = relPath.replace(/\\/g, '/');
zip.file(relPosixPath, fs.createReadStream(fullPath));
zip.file(posixify(relPath), fs.createReadStream(fullPath));
}
}
};
Expand Down
7 changes: 3 additions & 4 deletions src/convert/replacements.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,12 @@
*/
import { readFile } from 'node:fs/promises';
import { Transform, Readable } from 'node:stream';
import { sep, posix, join, isAbsolute } from 'node:path';
import { join, isAbsolute } from 'node:path';
import { Lifecycle, Messages, SfError, SfProject } from '@salesforce/core';
import { minimatch } from 'minimatch';
import { Env } from '@salesforce/kit';
import { ensureString, isString } from '@salesforce/ts-types';
import { posixify } from '../utils/path';
import { SourcePath } from '../common/types';
import { SourceComponent } from '../resolve/sourceComponent';
import { MarkedReplacement, ReplacementConfig, ReplacementEvent } from './types';
Expand Down Expand Up @@ -199,7 +200,7 @@ export const matchesFile =
(r: ReplacementConfig): boolean =>
// filenames will be absolute. We don't have convenient access to the pkgDirs,
// so we need to be more open than an exact match
(typeof r.filename === 'string' && posixifyPaths(filename).endsWith(r.filename)) ||
(typeof r.filename === 'string' && posixify(filename).endsWith(r.filename)) ||
(typeof r.glob === 'string' && minimatch(filename, `**/${r.glob}`));

/**
Expand Down Expand Up @@ -245,8 +246,6 @@ export const stringToRegex = (input: string): RegExp =>
// eslint-disable-next-line no-useless-escape
new RegExp(input.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&'), 'g');

export const posixifyPaths = (f: string): string => f.split(sep).join(posix.sep);

/** if replaceWithFile is present, resolve it to an absolute path relative to the projectdir */
const makeAbsolute =
(projectDir: string) =>
Expand Down
4 changes: 2 additions & 2 deletions src/convert/streams.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { createWriteStream, existsSync, promises as fsPromises } from 'graceful-
import { JsonMap } from '@salesforce/ts-types';
import { XMLBuilder } from 'fast-xml-parser';
import { Logger } from '@salesforce/core';
import { posixify } from '../utils/path';
import { SourceComponent } from '../resolve/sourceComponent';
import { SourcePath } from '../common/types';
import { XML_COMMENT_PROP_NAME, XML_DECL } from '../common/constants';
Expand Down Expand Up @@ -234,8 +235,7 @@ export class ZipWriter extends ComponentWriter {

public addToZip(contents: string | Readable | Buffer, path: SourcePath): void {
// Ensure only posix paths are added to zip files
const posixPath = path.replace(/\\/g, '/');
this.zip.file(posixPath, contents);
this.zip.file(posixify(path), contents);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import { JsonMap } from '@salesforce/ts-types';
import { createWriteStream } from 'graceful-fs';
import { Logger, Messages, SfError } from '@salesforce/core';
import { isEmpty } from '@salesforce/kit';
import { baseName } from '../../utils/path';
import { baseName, posixify } from '../../utils/path';
import { WriteInfo } from '../types';
import { SourceComponent } from '../../resolve/sourceComponent';
import { SourcePath } from '../../common/types';
Expand Down Expand Up @@ -59,8 +59,7 @@ export class StaticResourceMetadataTransformer extends BaseMetadataTransformer {
// have to walk the component content. Replacements only happen if set on the component.
for (const path of component.walkContent()) {
const replacementStream = getReplacementStreamForReadable(component, path);
const relPath = relative(content, path);
const relPosixPath = relPath.replace(/\\/g, '/');
const relPosixPath = posixify(relative(content, path));
zip.file(relPosixPath, replacementStream);
}

Expand Down
6 changes: 6 additions & 0 deletions src/registry/registryAccess.ts
Original file line number Diff line number Diff line change
Expand Up @@ -168,3 +168,9 @@ export class RegistryAccess {
}
}
}

/** decomposed and default types are `false`, everything else is true */
export const typeAllowsMetadataWithContent = (type: MetadataType): boolean =>
type.strategies?.adapter !== undefined && // another way of saying default
type.strategies.adapter !== 'decomposed' &&
type.strategies.adapter !== 'default';
203 changes: 82 additions & 121 deletions src/resolve/adapters/baseSourceAdapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,146 +5,104 @@
* For full license text, see LICENSE.txt file in the repo root or https://opensource.org/licenses/BSD-3-Clause
*/
import { basename, dirname, sep } from 'node:path';
import { Messages, SfError } from '@salesforce/core';
import { Lifecycle, Messages, SfError } from '@salesforce/core';
import { ensureString } from '@salesforce/ts-types';
import { MetadataXml, SourceAdapter } from '../types';
import { MetadataXml } from '../types';
import { parseMetadataXml, parseNestedFullName } from '../../utils/path';
import { ForceIgnore } from '../forceIgnore';
import { NodeFSTreeContainer, TreeContainer } from '../treeContainers';
import { SourceComponent } from '../sourceComponent';
import { SourcePath } from '../../common/types';
import { MetadataType } from '../../registry/types';
import { RegistryAccess } from '../../registry/registryAccess';
import { RegistryAccess, typeAllowsMetadataWithContent } from '../../registry/registryAccess';
import { FindRootMetadata, MaybeGetComponent } from './types';

Messages.importMessagesDirectory(__dirname);
const messages = Messages.loadMessages('@salesforce/source-deploy-retrieve', 'sdr');

export abstract class BaseSourceAdapter implements SourceAdapter {
protected type: MetadataType;
protected registry: RegistryAccess;
protected forceIgnore: ForceIgnore;
protected tree: TreeContainer;

/**
* Whether or not an adapter should expect a component to be in its own, self-named
* folder, including its root metadata xml file.
*/
protected ownFolder = false;
protected metadataWithContent = true;

public constructor(
type: MetadataType,
registry = new RegistryAccess(),
forceIgnore = new ForceIgnore(),
tree = new NodeFSTreeContainer()
) {
this.type = type;
this.registry = registry;
this.forceIgnore = forceIgnore;
this.tree = tree;
}

public getComponent(path: SourcePath, isResolvingSource = true): SourceComponent | undefined {
let rootMetadata = this.parseAsRootMetadataXml(path);
if (!rootMetadata) {
const rootMetadataPath = this.getRootMetadataXmlPath(path);
if (rootMetadataPath) {
rootMetadata = this.parseMetadataXml(rootMetadataPath);
}
}
if (rootMetadata && this.forceIgnore.denies(rootMetadata.path)) {
throw new SfError(
messages.getMessage('error_no_metadata_xml_ignore', [rootMetadata.path, path]),
'UnexpectedForceIgnore'
);
}

let component: SourceComponent | undefined;
if (rootMetadata) {
const name = calculateName(this.registry)(this.type)(rootMetadata);
component = new SourceComponent(
{
name,
type: this.type,
xml: rootMetadata.path,
parentType: this.type.folderType ? this.registry.getTypeByName(this.type.folderType) : undefined,
},
this.tree,
this.forceIgnore
);
/**
* If the path given to `getComponent` is the root metadata xml file for a component,
* parse the name and return it. This is an optimization to not make a child adapter do
* anymore work to find it.
*
* @param path File path of a metadata component
*/
export const parseAsRootMetadataXml = ({
type,
path,
}: {
type: MetadataType;
path: SourcePath;
}): MetadataXml | undefined => {
const metaXml = parseMetadataXml(path);
if (metaXml) {
let isRootMetadataXml = false;
if (type.strictDirectoryName) {
const parentPath = dirname(path);
const typeDirName = basename(type.inFolder ? dirname(parentPath) : parentPath);
const nameMatchesParent = basename(parentPath) === metaXml.fullName;
const inTypeDir = typeDirName === type.directoryName;
// if the parent folder name matches the fullName OR parent folder name is
// metadata type's directory name, it's a root metadata xml.
isRootMetadataXml = nameMatchesParent || inTypeDir;
} else {
isRootMetadataXml = true;
}
return isRootMetadataXml ? metaXml : undefined;
}

return this.populate(path, component, isResolvingSource);
const folderMetadataXml = parseAsFolderMetadataXml(path);
if (folderMetadataXml) {
return folderMetadataXml;
}

/**
* Control whether metadata and content metadata files are allowed for an adapter.
*/
public allowMetadataWithContent(): boolean {
return this.metadataWithContent;
if (!typeAllowsMetadataWithContent(type)) {
return parseAsContentMetadataXml(type)(path);
}
};

/**
* If the path given to `getComponent` is the root metadata xml file for a component,
* parse the name and return it. This is an optimization to not make a child adapter do
* anymore work to find it.
*
* @param path File path of a metadata component
*/
protected parseAsRootMetadataXml(path: SourcePath): MetadataXml | undefined {
const metaXml = this.parseMetadataXml(path);
if (metaXml) {
let isRootMetadataXml = false;
if (this.type.strictDirectoryName) {
const parentPath = dirname(path);
const typeDirName = basename(this.type.inFolder ? dirname(parentPath) : parentPath);
const nameMatchesParent = basename(parentPath) === metaXml.fullName;
const inTypeDir = typeDirName === this.type.directoryName;
// if the parent folder name matches the fullName OR parent folder name is
// metadata type's directory name, it's a root metadata xml.
isRootMetadataXml = nameMatchesParent || inTypeDir;
} else {
isRootMetadataXml = true;
}
return isRootMetadataXml ? metaXml : undefined;
}
/**
* Trim a path up until the root of a component's content. If the content is a file,
* the given path will be returned back. If the content is a folder, the path to that
* folder will be returned. Intended to be used exclusively for MixedContent types.
*
* @param path Path to trim
* @param type MetadataType to determine content for
*/
export const trimPathToContent =
(type: MetadataType) =>
(path: SourcePath): SourcePath => {
const pathParts = path.split(sep);
const typeFolderIndex = pathParts.lastIndexOf(type.directoryName);
const offset = type.inFolder ? 3 : 2;
return pathParts.slice(0, typeFolderIndex + offset).join(sep);
};

const folderMetadataXml = parseAsFolderMetadataXml(path);
if (folderMetadataXml) {
return folderMetadataXml;
export const getComponent: MaybeGetComponent =
(context) =>
({ type, path, metadataXml: findRootMetadata = defaultFindRootMetadata }) => {
// find rootMetadata
const metadataXml = typeof findRootMetadata === 'function' ? findRootMetadata(type, path) : findRootMetadata;
if (!metadataXml) {
void Lifecycle.getInstance().emitWarning(messages.getMessage('error_parsing_xml', [path, type.name]));
return;
}

if (!this.allowMetadataWithContent()) {
return parseAsContentMetadataXml(this.type)(path);
if (context.forceIgnore?.denies(metadataXml.path)) {
throw SfError.create({
message: messages.getMessage('error_no_metadata_xml_ignore', [metadataXml.path, path]),
name: 'UnexpectedForceIgnore',
});
}
}

// allowed to preserve API
// eslint-disable-next-line class-methods-use-this
protected parseMetadataXml(path: SourcePath): MetadataXml | undefined {
return parseMetadataXml(path);
}

/**
* Determine the related root metadata xml when the path given to `getComponent` isn't one.
*
* @param trigger Path that `getComponent` was called with
*/
protected abstract getRootMetadataXmlPath(trigger: SourcePath): SourcePath | undefined;

/**
* Populate additional properties on a SourceComponent, such as source files and child components.
* The component passed to `populate` has its fullName, xml, and type properties already set.
*
* @param component Component to populate properties on
* @param trigger Path that `getComponent` was called with
*/
protected abstract populate(
trigger: SourcePath,
component?: SourceComponent,
isResolvingSource?: boolean
): SourceComponent | undefined;
}
return new SourceComponent(
{
name: calculateName(context.registry)(type)(metadataXml),
type,
xml: metadataXml.path,
parentType: type.folderType ? context.registry.getTypeByName(type.folderType) : undefined,
},
context.tree,
context.forceIgnore
);
};

/**
* If the path given to `getComponent` serves as the sole definition (metadata and content)
Expand Down Expand Up @@ -192,6 +150,9 @@ const parseAsFolderMetadataXml = (fsPath: SourcePath): MetadataXml | undefined =
}
};

const defaultFindRootMetadata: FindRootMetadata = (type, path) =>
parseAsRootMetadataXml({ type, path }) ?? parseMetadataXml(path);

// Given a MetadataXml, build a fullName from the path and type.
const calculateName =
(registry: RegistryAccess) =>
Expand Down
Loading
Loading