Skip to content

Commit

Permalink
debug
Browse files Browse the repository at this point in the history
Signed-off-by: tnazarew <[email protected]>
  • Loading branch information
tnazarew committed Oct 24, 2024
1 parent dcaf71b commit c5048aa
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 5 deletions.
18 changes: 15 additions & 3 deletions .github/actions/get_openlineage_artifacts/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,18 @@ outputs:
runs:
using: "composite"
steps:
- name: create download dir in tmp
id: create-download-dir
shell: bash
run: mkdir -p /tmp/openlineage_snapshots

- name: Download OpenLineage artifacts from build
if: ${{ inputs.get-latest-snapshots == 'true' }}
id: download-snapshots
shell: bash
run: |
python ./scripts/get_latest_unreleased_artifact.py \
--path=./ \
--path=/tmp/openlineage_snapshots \
--skip-java=${{ inputs.skip-java }} \
--skip-spark=${{ inputs.skip-spark }} \
--skip-flink=${{ inputs.skip-flink }} \
Expand All @@ -88,6 +93,13 @@ runs:
--skip-gcs=${{ inputs.skip-gcs }} \
--skip-s3=${{ inputs.skip-s3 }}
- name: list downloads
id: list-downloads
shell: bash
run: |
echo "downloaded artifacts:"
echo "/tmp/openlineage_snapshots"
# artifacts snapshots are not published in latest build if no code was changed in them
# this checks which of not skipped artifacts are missing and marks them as 'to download from maven'
# if previous step was skipped, all are missing (besides skipped)
Expand All @@ -97,8 +109,8 @@ runs:
run: |
check_artifact_downloaded() {
if [ $2 = 'false' ]; then
if [ -f ./$1 ]; then
echo "$3=$(ls ./$1)" >> $GITHUB_OUTPUT
if [ -f /tmp/openlineage_snapshots/$1 ]; then
echo "$3=$(ls /tmp/openlineage_snapshots/$1)" >> $GITHUB_OUTPUT
else
echo "skip-$3=false" >> $GITHUB_OUTPUT
fi
Expand Down
3 changes: 1 addition & 2 deletions scripts/upload_file_to_gcs.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,8 @@ def main():
credentials_path = args.credentials

if not os.path.exists(local_file_path):
print(f"file {local_file_path} not found")
# Local file does not exist; do nothing
sys.exit(1)
sys.exit(0)

if not gcs_path.startswith('gs://'):
print("Error: GCS path must start with gs://")
Expand Down

0 comments on commit c5048aa

Please sign in to comment.