Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: 0.98.x tar.gz file size increased? #2599

Open
brandonburkett opened this issue Mar 26, 2024 · 10 comments
Open

[Bug]: 0.98.x tar.gz file size increased? #2599

brandonburkett opened this issue Mar 26, 2024 · 10 comments
Labels
🐛 bug Something isn't working

Comments

@brandonburkett
Copy link

brandonburkett commented Mar 26, 2024

Bug report

With version 0.97 and below, the tar.gz file size was around 135mb, but with 0.98 release, I see it is up to 400mb+. Our elastic beanstalk default /tmp only has 924M without attempting to give /tmp more space.

In short, php datadog-setup.php --php-bin=all is failing because /tmp is out of disk space while trying to untar, which was not a problem with 0.97 and below.

PHP version

8.2.15

Tracer or profiler version

0.98.x

Installed extensions

No response

Output of phpinfo()

No response

Upgrading from

No response

@brandonburkett brandonburkett added the 🐛 bug Something isn't working label Mar 26, 2024
@bwoebi
Copy link
Collaborator

bwoebi commented Mar 26, 2024

Hey @brandonburkett,

We've decided to keep full debug symbols of the rust code to enhance debuggability in case there are issues. However, this comes, as you noticed at the cost of a size increase.

To avoid the particular issue with /tmp space being exceeded, you can specify a different temp directory: php -d sys_temp_dir=/other/path datadog-setup.php --php-bin=all.

I'll keep this issue open so that we invest into splitting the .tar.gz up or using other strategies.

@morrisonlevi
Copy link
Collaborator

I slightly improved the situation in v1.1.0. Here are the sizes for the aarch64-linux-gnu artifacts over time, but there are similarly sized improvements for x86_64-linux-gnu:

dd-library-php-0.98.0-aarch64-linux-gnu.tar.gz  482 MB
dd-library-php-0.99.0-aarch64-linux-gnu.tar.gz  485 MB
dd-library-php-1.0.0-aarch64-linux-gnu.tar.gz   495 MB

dd-library-php-1.1.0-aarch64-linux-gnu.tar.gz   425 MB

It's still over 400 MiB, so we plan to continue working on this.

@awons
Copy link

awons commented Jun 18, 2024

We worked around it by copying only the necessary files and using them in our final Docker image with this script:

cd $(mktemp -d)
curl -Lo datadog-php-tracer.apk https://github.com/DataDog/dd-trace-php/releases/download/${DATADOG_TRACER_VERSION}/datadog-php-tracer_${DATADOG_TRACER_VERSION}_$(uname -m).apk
apk add datadog-php-tracer.apk --allow-untrusted
PHP_API=$(php -i | grep "PHP API" | awk '{split($0,a,"=>"); print a[2]}' | xargs)
DD_TRACER_EXTENSION="/opt/datadog-php/extensions/ddtrace-${PHP_API}-alpine.so"
for filename in /opt/datadog-php/extensions/*; do
  [ -e "${filename}" ] || continue
  if [ "${filename}" = "${DD_TRACER_EXTENSION}" ]; then continue; fi
  rm -f "${filename}"
done

mkdir /tmp/ddtracer
for filename in $(apk info -L datadog-php-tracer); do
  [ -e "/${filename}" ] || continue
  cp -p --parents "/${filename}" /tmp/ddtracer
done
cp -p --parents /usr/local/etc/php/conf.d/98-ddtrace.ini /tmp/ddtracer

We use in combination with COPY --from=ddtracer /tmp/ddtracer / but this could easily be adapted for use cases outside of Docker environment.

Our images are down to <70MB.

@Liphtier
Copy link

the Alpine apk installation leaves all unneded bundled extension binaries in place:

du -had1 /opt/datadog-php/extensions/
15.7M   /opt/datadog-php/extensions/ddtrace-20180731-alpine.so
16.5M   /opt/datadog-php/extensions/ddtrace-20230831.so
16.3M   /opt/datadog-php/extensions/ddtrace-20180731.so
15.8M   /opt/datadog-php/extensions/ddtrace-20230831-alpine.so
15.8M   /opt/datadog-php/extensions/ddtrace-20220829-alpine.so
16.0M   /opt/datadog-php/extensions/ddtrace-20180731-debug.so
1.6M    /opt/datadog-php/extensions/ddtrace-20121113-debug.so
1.6M    /opt/datadog-php/extensions/ddtrace-20100412-debug.so
16.4M   /opt/datadog-php/extensions/ddtrace-20200930.so
16.2M   /opt/datadog-php/extensions/ddtrace-20180731-zts.so
2.0M    /opt/datadog-php/extensions/ddtrace-20121113-zts.so
16.4M   /opt/datadog-php/extensions/ddtrace-20170718.so
2.1M    /opt/datadog-php/extensions/ddtrace-20121113.so
16.5M   /opt/datadog-php/extensions/ddtrace-20220829.so
16.0M   /opt/datadog-php/extensions/ddtrace-20160303-debug.so
15.7M   /opt/datadog-php/extensions/ddtrace-20160303-alpine.so
16.1M   /opt/datadog-php/extensions/ddtrace-20220829-debug.so
15.7M   /opt/datadog-php/extensions/ddtrace-20190902-alpine.so
16.4M   /opt/datadog-php/extensions/ddtrace-20210902-zts.so
15.8M   /opt/datadog-php/extensions/ddtrace-20210902-alpine.so
2.0M    /opt/datadog-php/extensions/ddtrace-20100412.so
15.7M   /opt/datadog-php/extensions/ddtrace-20170718-alpine.so
2.0M    /opt/datadog-php/extensions/ddtrace-20131106-alpine.so
16.2M   /opt/datadog-php/extensions/ddtrace-20170718-zts.so
2.0M    /opt/datadog-php/extensions/ddtrace-20121113-alpine.so
15.8M   /opt/datadog-php/extensions/ddtrace-20200930-alpine.so
16.1M   /opt/datadog-php/extensions/ddtrace-20190902-debug.so
16.4M   /opt/datadog-php/extensions/ddtrace-20230831-zts.so
16.0M   /opt/datadog-php/extensions/ddtrace-20170718-debug.so
2.1M    /opt/datadog-php/extensions/ddtrace-20131106.so
2.0M    /opt/datadog-php/extensions/ddtrace-20131106-zts.so
16.2M   /opt/datadog-php/extensions/ddtrace-20160303-zts.so
16.3M   /opt/datadog-php/extensions/ddtrace-20200930-zts.so
16.3M   /opt/datadog-php/extensions/ddtrace-20151012.so
16.4M   /opt/datadog-php/extensions/ddtrace-20220829-zts.so
16.5M   /opt/datadog-php/extensions/ddtrace-20210902.so
16.1M   /opt/datadog-php/extensions/ddtrace-20200930-debug.so
16.2M   /opt/datadog-php/extensions/ddtrace-20151012-zts.so
1.7M    /opt/datadog-php/extensions/ddtrace-20131106-debug.so
16.2M   /opt/datadog-php/extensions/ddtrace-20190902-zts.so
16.2M   /opt/datadog-php/extensions/ddtrace-20210902-debug.so
15.7M   /opt/datadog-php/extensions/ddtrace-20151012-alpine.so
16.4M   /opt/datadog-php/extensions/ddtrace-20190902.so
16.0M   /opt/datadog-php/extensions/ddtrace-20151012-debug.so
16.2M   /opt/datadog-php/extensions/ddtrace-20230831-debug.so
2.0M    /opt/datadog-php/extensions/ddtrace-20100412-zts.so
2.0M    /opt/datadog-php/extensions/ddtrace-20100412-alpine.so
16.3M   /opt/datadog-php/extensions/ddtrace-20160303.so
603.7M  /opt/datadog-php/extensions/

@razvanphp
Copy link

This is really unacceptable, our docker image with 0.89.0 is 455 Mb, 0.99.1 is 907 Mb, 1.2.0 is 1076 Mb, keeping everything else unchanged. We use .deb files from github releases.

BTW: maybe using https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php is a good alternative.

@bwoebi
Copy link
Collaborator

bwoebi commented Aug 26, 2024

@razvanphp If you use the docker image, you probably should download, use dpkg and remove the .deb file in the same dockerfile RUN line, to avoids storing the whole .deb file in the image itself.

@razvanphp
Copy link

razvanphp commented Aug 26, 2024

I do that already, still the sizes above. I think you should publish separate -dbg.deb packages instead.

the Alpine apk installation leaves all unneded bundled extension binaries in place:

BTW, .deb does the same:

root@server:/web# du -sh /opt/datadog-php/extensions/
231M	/opt/datadog-php/extensions/

@bwoebi
Copy link
Collaborator

bwoebi commented Aug 26, 2024

Oh, you are right, sorry. This is the .deb. I suppose we need to strip the extensions we don't install somehow.

The problem with separate -dbg.deb packages is that it's going to defeat the point of crashtracking which #2763 will introduce: we want to automatically collect fully symbolicated dumps (meaning we need at least line tables).
The primary problem here lies with the linker - rust depends on the linker to strip away unused code, but debug info of unused code stays in the binaries and that's blowing up the binary sizes :-(

@razvanphp
Copy link

razvanphp commented Aug 27, 2024

While I totally understand the reasoning why you would need this, please imagine what this does for an infrastructure where we deploy thousands of containers, each of them with a size of 1 GB; not counting the repo traffic cost, the time it takes for one container to come up is exponential with the size.

@bwoebi
Copy link
Collaborator

bwoebi commented Nov 18, 2024

datadog-setup.php is now split in individual small packages, downloading less than 50 MB for single version installs.
We probably need to set this up for deb and rpm and apk too, but for these we'll likely have to move to proper repositories instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

6 participants