Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase to v3.4.6 #50

Merged
merged 997 commits into from
May 12, 2023
Merged

Rebase to v3.4.6 #50

merged 997 commits into from
May 12, 2023

Conversation

dscho
Copy link
Member

@dscho dscho commented May 12, 2023

After deploying git-for-windows/MSYS2-packages#102, we can finally upgrade Git for Windows' MSYS2 runtime to the current version.

The long-delayed upgrade of Git for Windows' fork of the MSYS2 runtime to v3.4.* is finally here. And it comes with quite a few changes. The following two patches made it into the MSYS2 runtime proper:

The following two patches made it even further upstream, into Cygwin proper:

Finally, we backported the commit 4d62197, so naturally it is no longer necessary to carry it in Git for Windows' fork.

Range-diff (limited to the patches Git for Windows carries on top of the MSYS2 runtime)

Note: the range-diff does not handle the change in "Make paths' WCS->MBS conversion explicit" gracefully where the order of the files was changed because the wchar.h file was moved in Cygwin's source code.

$ git range-diff --creation-factor=99 \
  git-for-windows/main^{/^Merge.branch.*msys2}..git-for-windows/main \
  rebase-to-v3.4.6^{/^Merge.branch.*msys2}..rebase-to-v3.4.6
  • 1: 3e9c003 < -: ---------- Pass environment variables with empty values

  • 2: 1e0136e ! 1: de4ab2e Handle 8-bit characters under LOCALE=C

    @@ newlib/libc/stdlib/mbtowc_r.c: __ascii_mbtowc (struct _reent *r,
     +#ifdef STRICTLY_7BIT_ASCII
        if ((wchar_t)*t >= 0x80)
          {
    -       r->_errno = EILSEQ;
    +       _REENT_ERRNO(r) = EILSEQ;
     
      ## newlib/libc/stdlib/wctomb_r.c ##
     @@ newlib/libc/stdlib/wctomb_r.c: __ascii_wctomb (struct _reent *r,
  • 3: a5012b9 ! 2: 63b328e Mention the extremely useful small_printf() function

    @@ Commit message
     
         Signed-off-by: Johannes Schindelin <[email protected]>
     
    - ## winsup/cygwin/how-to-debug-cygwin.txt ##
    -@@ winsup/cygwin/how-to-debug-cygwin.txt: set CYGWIN_DEBUG=cat.exe:gdb.exe
    + ## winsup/cygwin/DevDocs/how-to-debug-cygwin.txt ##
    +@@ winsup/cygwin/DevDocs/how-to-debug-cygwin.txt: set CYGWIN_DEBUG=cat.exe:gdb.exe
         program will crash, probably in small_printf.  At that point, a 'bt'
         command should show you the offending call to strace_printf with the
         improper format string.
  • 4: d2456ea < -: ---------- Add a helpful debug message for posix-to-windows conversion

  • 5: 0b2d287 < -: ---------- Stop assuming that there are no spaces in POSIX-style paths

  • 6: 78fbdbb < -: ---------- Avoid unnecessary recursion in find_path_start_and_type()

  • 7: 707f9b2 = 3: f27f65e Leave arguments starting with a tilde or quote alone

  • 8: c312f3d = 4: 9780f9c Leave Git's :name and :/message arguments alone, please

  • 9: 99a84e0 = 5: b63b228 Leave paths containing any special characters alone

  • 10: 6efd127 = 6: 5824c7a Leave paths containing '/~' alone

  • 11: 4076235 = 7: 43f2118 Skip posix-to-windows conversion when '::' is seen

  • 12: 12461ac = 8: 1c5c1da Also leave Git's :./ syntax alone

  • 13: aeb2e42 = 9: 07b11c9 Arguments starting with '@@' are no paths

  • 14: 9f3e592 = 10: 07fd72e Prevent scp-style arguments from being mangled

  • 16: 92a0032 = 11: e01713d Fixed path converting with non ascii char.

  • 18: f874ac1 = 12: 09e6db7 path-conversion: Introduce ability to switch off conversion.

  • 15: ad711d9 ! 13: 81570ce Allow overriding the home directory via the HOME variable

    @@ Commit message
     
         Signed-off-by: Johannes Schindelin <[email protected]>
     
    - ## winsup/cygwin/cygheap.h ##
    -@@ winsup/cygwin/cygheap.h: public:
    + ## winsup/cygwin/local_includes/cygheap.h ##
    +@@ winsup/cygwin/local_includes/cygheap.h: public:
          NSS_SCHEME_UNIX,
          NSS_SCHEME_DESC,
          NSS_SCHEME_PATH,
  • 17: 90d8aad = 14: 2078dff Respect db_home setting even for the SYSTEM account

  • 19: 4935495 = 15: dda3c93 Respect the db_home: env setting under more circumstances

  • 21: 99c3c01 ! 16: 42b0ac9 Make paths' WCS->MBS conversion explicit

    @@ winsup/cygwin/external.cc: fillout_pinfo (pid_t pid, int winpid)
      	  ep.version = EXTERNAL_PINFO_VERSION;
      
     
    - ## winsup/cygwin/fhandler_disk_file.cc ##
    -@@ winsup/cygwin/fhandler_disk_file.cc: fhandler_disk_file::readdir_helper (DIR *dir, dirent *de, DWORD w32_err,
    + ## winsup/cygwin/fhandler/disk_file.cc ##
    +@@ winsup/cygwin/fhandler/disk_file.cc: fhandler_disk_file::readdir_helper (DIR *dir, dirent *de, DWORD w32_err,
      	  char *p = stpcpy (file, pc.get_posix ());
      	  if (p[-1] != '/')
      	    *p++ = '/';
    @@ winsup/cygwin/fhandler_disk_file.cc: fhandler_disk_file::readdir_helper (DIR *di
      			fname->Buffer, fname->Length / sizeof (WCHAR));
      	  path_conv fpath (file, PC_SYM_NOFOLLOW);
      	  if (fpath.issymlink ())
    -@@ winsup/cygwin/fhandler_disk_file.cc: fhandler_disk_file::readdir_helper (DIR *dir, dirent *de, DWORD w32_err,
    +@@ winsup/cygwin/fhandler/disk_file.cc: fhandler_disk_file::readdir_helper (DIR *dir, dirent *de, DWORD w32_err,
      	}
          }
      
    @@ winsup/cygwin/fhandler_disk_file.cc: fhandler_disk_file::readdir_helper (DIR *di
      
        /* Don't try to optimize relative to dir->__d_position.  On several
     
    - ## winsup/cygwin/fhandler_netdrive.cc ##
    -@@ winsup/cygwin/fhandler_netdrive.cc: fhandler_netdrive::readdir (DIR *dir, dirent *de)
    + ## winsup/cygwin/fhandler/netdrive.cc ##
    +@@ winsup/cygwin/fhandler/netdrive.cc: fhandler_netdrive::readdir (DIR *dir, dirent *de)
      	  tp.u_get (&ds);
      	  RtlInitUnicodeString (&ss, bs);
      	  RtlDowncaseUnicodeString (&ds, &ss, FALSE);
    @@ winsup/cygwin/fhandler_netdrive.cc: fhandler_netdrive::readdir (DIR *dir, dirent
      	  /* We can't trust remote inode numbers of only 32 bit.  That means,
      	     remote NT4 NTFS, as well as shares of Samba version < 3.0. */
     
    - ## winsup/cygwin/fhandler_process.cc ##
    -@@ winsup/cygwin/fhandler_process.cc: static off_t
    + ## winsup/cygwin/fhandler/process.cc ##
    +@@ winsup/cygwin/fhandler/process.cc: static off_t
      format_process_winexename (void *data, char *&destbuf)
      {
        _pinfo *p = (_pinfo *) data;
    @@ winsup/cygwin/fhandler_process.cc: static off_t
        return len;
      }
      
    -@@ winsup/cygwin/fhandler_process.cc: peb_teb_rinse_repeat:
    +@@ winsup/cygwin/fhandler/process.cc: peb_teb_rinse_repeat:
      		      drive_maps.fixup_if_match (msi->SectionFileName.Buffer);
      		  if (mount_table->conv_to_posix_path (dosname,
      						       posix_modname, 0))
     -		    sys_wcstombs (posix_modname, NT_MAX_PATH, dosname);
     +		    sys_wcstombs_path (posix_modname, NT_MAX_PATH, dosname);
    - 		  stat64 (posix_modname, &st);
    + 		  stat (posix_modname, &st);
      		}
      	      else if (!threads.fill_if_match (cur.abase, mb.Type,
    -@@ winsup/cygwin/fhandler_process.cc: format_process_stat (void *data, char *&destbuf)
    +@@ winsup/cygwin/fhandler/process.cc: format_process_stat (void *data, char *&destbuf)
        else
          {
            PWCHAR last_slash = wcsrchr (p->progname, L'\\');
    @@ winsup/cygwin/fhandler_process.cc: format_process_stat (void *data, char *&destb
      		    last_slash ? last_slash + 1 : p->progname);
            int len = strlen (cmd);
            if (len > 4)
    -@@ winsup/cygwin/fhandler_process.cc: format_process_status (void *data, char *&destbuf)
    -   size_t vmsize = 0, vmrss = 0, vmdata = 0, vmlib = 0, vmtext = 0, vmshare = 0;
    +@@ winsup/cygwin/fhandler/process.cc: format_process_status (void *data, char *&destbuf)
    +   bool fetch_siginfo = false;
      
        PWCHAR last_slash = wcsrchr (p->progname, L'\\');
     -  sys_wcstombs (cmd, NAME_MAX + 1, last_slash ? last_slash + 1 : p->progname);
    @@ winsup/cygwin/fhandler_process.cc: format_process_status (void *data, char *&des
        if (len > 4)
          {
     
    - ## winsup/cygwin/fhandler_procsys.cc ##
    -@@ winsup/cygwin/fhandler_procsys.cc: fhandler_procsys::fill_filebuf ()
    + ## winsup/cygwin/fhandler/procsys.cc ##
    +@@ winsup/cygwin/fhandler/procsys.cc: fhandler_procsys::fill_filebuf ()
        NtClose (h);
        if (!NT_SUCCESS (status))
          goto unreadable;
    @@ winsup/cygwin/fhandler_procsys.cc: fhandler_procsys::fill_filebuf ()
        while ((fnamep = strchr (fnamep, '\\')))
          *fnamep = '/';
        return true;
    -@@ winsup/cygwin/fhandler_procsys.cc: fhandler_procsys::readdir (DIR *dir, dirent *de)
    +@@ winsup/cygwin/fhandler/procsys.cc: fhandler_procsys::readdir (DIR *dir, dirent *de)
      	res = ENMFILE;
            else
      	{
    @@ winsup/cygwin/fhandler_procsys.cc: fhandler_procsys::readdir (DIR *dir, dirent *
      	  if (RtlEqualUnicodeString (&dbi->ObjectTypeName, &ro_u_natdir, FALSE))
      	    de->d_type = DT_DIR;
     
    + ## winsup/cygwin/local_includes/wchar.h ##
    +@@ winsup/cygwin/local_includes/wchar.h: extern size_t _sys_wcstombs_alloc (char **dst_p, int type, const wchar_t *src,
    + 				   size_t nwc, bool is_path);
    + 
    + static inline size_t
    +-sys_wcstombs (char *dst, size_t len, const wchar_t * src,
    +-	      size_t nwc = (size_t) -1)
    ++sys_wcstombs_path (char *dst, size_t len, const wchar_t * src,
    ++		   size_t nwc = (size_t) -1)
    + {
    +   return _sys_wcstombs (dst, len, src, nwc, true);
    + }
    + 
    + static inline size_t
    +-sys_wcstombs_no_path (char *dst, size_t len, const wchar_t * src,
    +-		      size_t nwc = (size_t) -1)
    ++sys_wcstombs (char *dst, size_t len, const wchar_t * src,
    ++	      size_t nwc = (size_t) -1)
    + {
    +   return _sys_wcstombs (dst, len, src, nwc, false);
    + }
    + 
    + static inline size_t
    +-sys_wcstombs_alloc (char **dst_p, int type, const wchar_t *src,
    +-		    size_t nwc = (size_t) -1)
    ++sys_wcstombs_alloc_path (char **dst_p, int type, const wchar_t *src,
    ++			 size_t nwc = (size_t) -1)
    + {
    +   return _sys_wcstombs_alloc (dst_p, type, src, nwc, true);
    + }
    + 
    + static inline size_t
    +-sys_wcstombs_alloc_no_path (char **dst_p, int type, const wchar_t *src,
    +-			    size_t nwc = (size_t) -1)
    ++sys_wcstombs_alloc (char **dst_p, int type, const wchar_t *src,
    ++		    size_t nwc = (size_t) -1)
    + {
    +   return _sys_wcstombs_alloc (dst_p, type, src, nwc, false);
    + }
    +
      ## winsup/cygwin/mount.cc ##
     @@ winsup/cygwin/mount.cc: fs_info::update (PUNICODE_STRING upath, HANDLE in_vol)
          {
    @@ winsup/cygwin/uinfo.cc: cygheap_pwdgrp::get_shell (cyg_ldap *pldap, cygpsid &sid
      		}
      	    }
      	  break;
    -
    - ## winsup/cygwin/wchar.h ##
    -@@ winsup/cygwin/wchar.h: extern size_t _sys_wcstombs_alloc (char **dst_p, int type, const wchar_t *src,
    - 				   size_t nwc, bool is_path);
    - 
    - static inline size_t
    --sys_wcstombs (char *dst, size_t len, const wchar_t * src,
    --	      size_t nwc = (size_t) -1)
    -+sys_wcstombs_path (char *dst, size_t len, const wchar_t * src,
    -+		   size_t nwc = (size_t) -1)
    - {
    -   return _sys_wcstombs (dst, len, src, nwc, true);
    - }
    - 
    - static inline size_t
    --sys_wcstombs_no_path (char *dst, size_t len, const wchar_t * src,
    --		      size_t nwc = (size_t) -1)
    -+sys_wcstombs (char *dst, size_t len, const wchar_t * src,
    -+	      size_t nwc = (size_t) -1)
    - {
    -   return _sys_wcstombs (dst, len, src, nwc, false);
    - }
    - 
    - static inline size_t
    --sys_wcstombs_alloc (char **dst_p, int type, const wchar_t *src,
    --		    size_t nwc = (size_t) -1)
    -+sys_wcstombs_alloc_path (char **dst_p, int type, const wchar_t *src,
    -+			 size_t nwc = (size_t) -1)
    - {
    -   return _sys_wcstombs_alloc (dst_p, type, src, nwc, true);
    - }
    - 
    - static inline size_t
    --sys_wcstombs_alloc_no_path (char **dst_p, int type, const wchar_t *src,
    --			    size_t nwc = (size_t) -1)
    -+sys_wcstombs_alloc (char **dst_p, int type, const wchar_t *src,
    -+		    size_t nwc = (size_t) -1)
    - {
    -   return _sys_wcstombs_alloc (dst_p, type, src, nwc, false);
    - }
  • 20: 07e3f3f = 17: 0a1c226 Allow native symlinks to non-existing targets in 'nativestrict' mode

  • 22: d6849e0 = 18: 552263b Use MB_CUR_MAX == 6 by default

  • 23: 1fbb6ee ! 19: 3d6e236 Change the default base address for x86_64

    @@ Commit message
         Signed-off-by: Johannes Schindelin <[email protected]>
         Signed-off-by: Mikael Larsson <[email protected]>
     
    - ## winsup/cygwin/x86_64.din ##
    + ## winsup/cygwin/cygwin.din ##
     @@
     -LIBRARY "msys-2.0.dll" BASE=0x180040000
     +LIBRARY "msys-2.0.dll" BASE=0x210040000
      
      EXPORTS
    - #Exported variables
    + # Exported variables
  • 24: 54f02c5 = 20: 0438c97 Do not try to sync with Cygwin

  • 25: 4d62197 < -: ---------- Cygwin: console: Fix hangup of less on quit after the window is resized.

  • 26: 65ce633 < -: ---------- Avoid sharing cygheaps across Cygwin versions

  • 27: 4704d29 < -: ---------- dumper: avoid linker problem when libbfd depends on libsframe

  • 28: f5f594c ! 21: 53c11c5 amend! Avoid sharing cygheaps across Cygwin versions

    @@ Metadata
     Author: Johannes Schindelin <[email protected]>
     
      ## Commit message ##
    -    amend! Avoid sharing cygheaps across Cygwin versions
    -
         Avoid sharing cygheaps across Cygwin versions
     
         It frequently leads to problems when trying, say, to call from Git for
    @@ winsup/configure.ac: AC_CHECK_TOOL(RANLIB, ranlib, ranlib)
     +AC_SUBST(MSYS2_RUNTIME_COMMIT_HEX)
     +
      AC_ARG_ENABLE(debugging,
    - [ --enable-debugging		Build a cygwin DLL which has more consistency checking for debugging],
    + [AS_HELP_STRING([--enable-debugging],[Build a cygwin DLL which has more consistency checking for debugging])],
      [case "${enableval}" in
     
      ## winsup/cygwin/Makefile.am ##
    @@ winsup/cygwin/dcrt0.cc: get_cygwin_startup_info ()
        child_info *res = (child_info *) si.lpReserved2;
      
        if (si.cbReserved2 < EXEC_MAGIC_SIZE || !res
    --      || res->intro != PROC_MAGIC_GENERIC || res->magic != (CHILD_INFO_MAGIC ^ CYGWIN_VERSION_DLL_COMBINED))
    +-      || res->intro != PROC_MAGIC_GENERIC || res->magic != CHILD_INFO_MAGIC)
     +      || res->intro != PROC_MAGIC_GENERIC || res->magic != (CHILD_INFO_MAGIC ^ MSYS2_RUNTIME_COMMIT_HEX))
          {
            strace.activate (false);
    @@ winsup/cygwin/dcrt0.cc: get_cygwin_startup_info ()
     
      ## winsup/cygwin/sigproc.cc ##
     @@ winsup/cygwin/sigproc.cc: int child_info::retry_count = 0;
    -    by fork/spawn/exec. */
      child_info::child_info (unsigned in_cb, child_info_types chtype,
      			bool need_subproc_ready):
    --  cb (in_cb), intro (PROC_MAGIC_GENERIC), magic (CHILD_INFO_MAGIC ^ CYGWIN_VERSION_DLL_COMBINED),
    -+  cb (in_cb), intro (PROC_MAGIC_GENERIC), magic (CHILD_INFO_MAGIC ^ MSYS2_RUNTIME_COMMIT_HEX),
    -   type (chtype), cygheap (::cygheap), cygheap_max (::cygheap_max),
    -   flag (0), retry (child_info::retry_count), rd_proc_pipe (NULL),
    -   wr_proc_pipe (NULL), sigmask (_my_tls.sigmask)
    +   msv_count (0), cb (in_cb), intro (PROC_MAGIC_GENERIC),
    +-  magic (CHILD_INFO_MAGIC), type (chtype), cygheap (::cygheap),
    ++  magic (CHILD_INFO_MAGIC ^ MSYS2_RUNTIME_COMMIT_HEX), type (chtype), cygheap (::cygheap),
    +   cygheap_max (::cygheap_max), flag (0), retry (child_info::retry_count),
    +   rd_proc_pipe (NULL), wr_proc_pipe (NULL), sigmask (_my_tls.sigmask)
    + {
  • 29: bec3d60 ! 22: ca20b5d uname: report msys2-runtime commit hash, too

    @@ winsup/cygwin/Makefile.am: if TARGET_X86_64
      COMMON_CFLAGS += $(VERSION_CFLAGS)
      
      AM_CFLAGS=$(cflags_common) $(COMMON_CFLAGS)
    -@@ winsup/cygwin/Makefile.am: src_files := $(foreach dir,$(dirs),$(find_src_files))
    - version.cc: mkvers.sh include/cygwin/version.h winver.rc $(src_files)
    +@@ winsup/cygwin/Makefile.am: uname_version.c: .FORCE
    + version.cc: scripts/mkvers.sh include/cygwin/version.h winver.rc $(src_files)
      	@echo "Making version.cc and winver.o";\
      	export CC="$(CC)";\
     -	/bin/sh $(word 1,$^) $(word 2,$^) $(word 3,$^) $(WINDRES) $(CFLAGS)
    @@ winsup/cygwin/Makefile.am: src_files := $(foreach dir,$(dirs),$(find_src_files))
      winver.o: version.cc
      
     
    - ## winsup/cygwin/mkvers.sh ##
    -@@ winsup/cygwin/mkvers.sh: incfile="$1"; shift
    + ## winsup/cygwin/scripts/mkvers.sh ##
    +@@ winsup/cygwin/scripts/mkvers.sh: incfile="$1"; shift
      rcfile="$1"; shift
      windres="$1"; shift
      iflags=
    @@ winsup/cygwin/mkvers.sh: incfile="$1"; shift
      # Find header file locations
      while [ -n "$*" ]; do
        case "$1" in
    -@@ winsup/cygwin/mkvers.sh: while [ -n "$*" ]; do
    +@@ winsup/cygwin/scripts/mkvers.sh: while [ -n "$*" ]; do
          shift
          iflags="$iflags -I$1"
            ;;
    @@ winsup/cygwin/mkvers.sh: while [ -n "$*" ]; do
        esac
        shift
      done
    -@@ winsup/cygwin/mkvers.sh: then
    +@@ winsup/cygwin/scripts/mkvers.sh: then
        cvs_tag="$(echo $wv_cvs_tag | sed -e 's/-branch.*//')"
        cygwin_ver="$cygwin_ver-$cvs_tag"
      fi
    @@ winsup/cygwin/mkvers.sh: then
     
      ## winsup/cygwin/uname.cc ##
     @@ winsup/cygwin/uname.cc: uname_x (struct utsname *name)
    -       cygwin_gethostname (buf, sizeof buf - 1);
    -       strncat (name->nodename, buf, sizeof (name->nodename) - 1);
    -       /* release */
    --      __small_sprintf (name->release, "%d.%d.%d-%d.",
    -+      __small_sprintf (name->release, "%d.%d.%d-%s-%d.",
    + 	snprintf (name->release, _UTSNAME_LENGTH, "%s.%s",
    + 		  uname_dev_version, name->machine);
    +       else
    +-	__small_sprintf (name->release, "%d.%d.%d-api-%d.%s",
    ++	__small_sprintf (name->release, "%d.%d.%d-%s-api-%d.%s",
    + 			 cygwin_version.dll_major / 1000,
    + 			 cygwin_version.dll_major % 1000,
    + 			 cygwin_version.dll_minor,
    ++			 MSYS2_RUNTIME_COMMIT_SHORT,
    + 			 cygwin_version.api_minor,
    + 			 name->machine);
    + #endif
    +@@ winsup/cygwin/uname.cc: uname (struct utsname *in_name)
    +       cygwin_gethostname (name->nodename, sizeof (name->nodename) - 1);
    + 
    +       /* Cygwin dll release */
    +-      __small_sprintf (name->release, "%d.%d.%d(%d.%d/%d/%d)",
    ++      __small_sprintf (name->release, "%d.%d.%d(%d.%d/%d/%d/%s)",
      		       cygwin_version.dll_major / 1000,
      		       cygwin_version.dll_major % 1000,
      		       cygwin_version.dll_minor,
    -+		       MSYS2_RUNTIME_COMMIT_SHORT,
    - 		       cygwin_version.api_minor);
    -       /* version */
    -       stpcpy (name->version, cygwin_version.dll_build_date);
    + 		       cygwin_version.api_major,
    + 		       cygwin_version.api_minor,
    + 		       cygwin_version.shared_data,
    +-		       cygwin_version.mount_registry);
    ++		       cygwin_version.mount_registry,
    ++		       MSYS2_RUNTIME_COMMIT_SHORT);
    + 
    +       /* Cygwin "version" aka build date */
    +       strcpy (name->version, cygwin_version.dll_build_date);

github-cygwin and others added 30 commits August 4, 2022 22:13
and drop unused prototypes from ntdll.h.

Signed-off-by: Corinna Vinschen <[email protected]>
…points

mount_info::get_mounts_here used RtlCreateUnicodeStringFromAsciiz
which translates bytes into wide chars verbatim.

Create a new function sys_mbstouni_alloc which can be used from
mount_info::get_mounts_here to convert multibyte mount point
strings to UNICODE_STRINGS in a locale-aware way.

For symmetry, create a function mount_info::free_mounts_here,
so the knwoledge how to free the UNICODE_STRING buffers is
encapsulated in the same class.

Signed-off-by: Corinna Vinschen <[email protected]>
- Previsouly, there was potential risk of buffer over run in
  is_console_app(). This patch fixes the issue.
This function is just bad.  It really only works for ASCII
chars, everything else is broken after the conversion.

Introduce new helper function sys_mbstouni to replace
RtlCreateUnicodeStringFromAsciiz in hash_path_name.

Signed-off-by: Corinna Vinschen <[email protected]>
…gheap

That *should* be slightly faster than allocating on the user heap.

Signed-off-by: Corinna Vinschen <[email protected]>
Create subdirs and move files accordingly:

- DevDocs:  doc files
- fhandler: fhandler sources, split fhandler.cc into base.cc and null.cc
- local_includes: local include files
- scripts:  scripts called during build
- sec:      security sources

Signed-off-by: Corinna Vinschen <[email protected]>
Add the new fhandler and sec subdirs.
move the assembler memset and memcpy functions into their own
assembler files.

Signed-off-by: Corinna Vinschen <[email protected]>
So far, wmemset used the C implemantation from newlib.  Let's use
the optimized assembler code instead.

Signed-off-by: Corinna Vinschen <[email protected]>
We don't need a target-specific DEF file anymore

Signed-off-by: Corinna Vinschen <[email protected]>
This is a remnant from 32 bit times, mindlessly copied into
the 64 bit export table.

Signed-off-by: Corinna Vinschen <[email protected]>
provide entire internal and external pthread API from inside the
same file.

While I dislike to have another even larger file, this is basically
cleaning up the source and grouping the external API into useful
chunks. Splitting the file cleanly is tricky due to usage of inline
methods is_good_object and verifyable_object_isvalid.

Signed-off-by: Corinna Vinschen <[email protected]>
Currently it is possible for symlink_info::check to return -1 in case
we're searching for foo and find foo.lnk that is not a Cygwin symlink.
This contradicts the new meaning attached to a negative return value
in commit 19d59ce.  Fix this by setting "res" to 0 at the beginning
of the main loop and not seting it to -1 later.

Also fix the commentary preceding the function definition to reflect
the current behavior.

Addresses: https://cygwin.com/pipermail/cygwin/2022-August/252030.html
They never worked as desired anyway. Use the object files.

Signed-off-by: Corinna Vinschen <[email protected]>
move it to mm/mmap.cc which uses it exclusively.

Signed-off-by: Corinna Vinschen <[email protected]>
It's used in malloc_init only and we never need it anywhere else,
hopefully.

Signed-off-by: Corinna Vinschen <[email protected]>
It's used in this file only anyway, so make it static inline.

Signed-off-by: Corinna Vinschen <[email protected]>
Rename CygwinCreateThread to create_posix_thread and move
from miscfuncs.cc to create_posix_thread.cc, inbcluding all
related functions.  Analogue for the prototypes.

Signed-off-by: Corinna Vinschen <[email protected]>
This way, the sem API is all in the same place, even if the
underlying semaphore class is still in thread.cc.

Signed-off-by: Corinna Vinschen <[email protected]>
This simple testcase:

  locale_t st = newlocale(LC_ALL_MASK, "C", (locale_t)0);
  locale_t st2 = newlocale(LC_CTYPE_MASK, "en_US.UTF-8", st);

is sufficient to reproduce a crash in _newlocale_r.  After the first call
to newlocale, `st' points to __C_locale, which is const.  When using `st'
as locale base in the second call, _newlocale_r tries to set pointers
inside base to NULL.  This is bad if base is __C_locale, obviously.

Add a test to avoid trying to overwrite pointer values inside base if
base is __C_locale.

Signed-off-by: Corinna Vinschen <[email protected]>
This patch makes syscalls for SH architecture respecting the global option
"--disable-newlib-supplied-syscalls". This is useful when a bare-metal
toolchain is needed.

Signed-off-by: Yilin Sun <[email protected]>
Signed-off-by: Corinna Vinschen <[email protected]>
Signed-off-by: Corinna Vinschen <[email protected]>
CloseHandle gets redefined to a macro calling an internal function
in debug.h when building with -DDEBUGGING, but profiler has no access
to that function.

Signed-off-by: Corinna Vinschen <[email protected]>
dscho and others added 20 commits May 12, 2023 15:52
An argument like [email protected]:/tmp/ is not something we should convert
into a Windows path; Use the absence of a slash before the colon as a
tell-tale that it is *not* a POSIX path list (exception: if the part
left of the colon is "." or "..").

Signed-off-by: Johannes Schindelin <[email protected]>
When a non ascii char is at the beginning of a path the current conversion
destroys the path. This fix will prevent this with an extra check for
non-ascii UTF-8 characters.

Helped-by: Johannes Schindelin <[email protected]>
Signed-off-by: 마누엘 <[email protected]>
When calling windows native apps from MSYS2, the runtime tries to convert
commandline arguments by a specific set of rules. See [MinGW wiki]
(http://www.mingw.org/wiki/Posix_path_conversion).

If the user does not want that behavior on a big scale, e.g. inside a bash
script, the user can now set the the environment variable
`MSYS_NO_PATHCONV` when calling native windows commands.

Signed-off-by: 마누엘 <[email protected]>
In Git for Windows, it is a well-established technique to use the
`$HOME` variable to define where the current user's home directory is,
falling back to `$HOMEDRIVE$HOMEPATH` and `$USERPROFILE`.

This strategy is particular important when Cygwin, or command-line
programs depending on the HOME variable, cannot cope with the Windows'
idea of the user's home directory e.g. when it is set to a hidden
directory via an UNC path (\\share\some\hidden\folder$).

Of course this strategy needs to be opt-in. For that reason, this
strategy is activated via the `env` keyword in the `db_home` line in
`/etc/nsswitch.conf`.

Signed-off-by: Johannes Schindelin <[email protected]>
We should not blindly set the home directory of the SYSTEM account to
/home/SYSTEM, especially not when that value disagrees with what is
configured via the `db_home` line in the `/etc/nsswitch.conf` file.

This fixes git-for-windows/git#435

Signed-off-by: Johannes Schindelin <[email protected]>
In particular when we cannot figure out a uid for the current user, we
should still respect the `db_home: env` setting. Such a situation occurs
for example when the domain returned by `LookupAccountSid()` is not our
machine name and at the same time our machine is no domain member: In
that case, we have nobody to ask for the POSIX offset necessary to come
up with the uid.

It is important that even in such cases, the HOME environment variable
can be used to override the home directory, e.g. when Git for Windows is
used by an account that was generated on the fly, e.g. for transient use
in a cloud scenario.

Reported by David Ebbo.

Signed-off-by: Johannes Schindelin <[email protected]>
	* dcrt0.cc (dll_crt0_1), dtable.cc (handle_to_fn), environ.cc
	(environ_init, getwinenveq, build_env), external.cc
	(fillout_pinfo), fhandler_disk_file.cc (__DIR_mounts::eval_ino,
	fhandler_disk_file::readdir_helper), fhandler_netdrive.cc
	(fhandler_netdrive::readdir), fhandler_process.cc
	(format_process_winexename, format_process_maps,
	format_process_stat, format_process_status), fhandler_procsys.cc
	(fill_filebuf, fhandler_procsys::readdir), mount.cc
	(fs_info::update, mount_info::create_root_entry,
	mount_info::conv_to_posix_path, mount_info::from_fstab_line),
	nlsfuncs.cc (internal_setlocale), path.cc (path_conv::check,
	sysmlink_info::check_shortcut, symlink_info::check_sysfile,
	symlink_info::check_reparse_point,
	symlink_info::check_nfs_symlink, cygwin_conv_path,
	cygwin_conv_path_list, cwdstuff::get_error_desc, cwdstuff::get),
	strfuncs.cc (sys_wcstombs_no_path, sys_wcstombs_alloc_no_path),
	uinfo.cc (ontherange, fetch_from_path, cygheap_pwdgrp::get_home,
	cygheap_pwdgrp::get_shell, cygheap_pwdgrp::get_gecos), wchar.h
	(sys_wcstombs_no_path, sys_wcstombs_alloc_no_path): Convert call
	sites of the sys_wcstombs*() family to specify explicitly when the
	parameter refers to a path or file name, to avoid future
	misconversions.

Detailed explanation:

The sys_wcstombs() function contains special handling for paths/file
names, to work around file name restriction on Windows that are
unexpected in the POSIX context of Cygwin.

We actually do not want that special handling for WCS strings that
do *not* refer to paths or file names.

Neither do we want to convert those special file names unless they come
from inside Cygwin: if the source of the string value is the Windows API,
we *know* it cannot be such a special file name because Windows itself
would not be able to handle it in the way Cygwin does.

So let's switch the previous sys_wcstombs()/sys_wcstombs_no_path() (and
the *_alloc* variant) around to sys_wcstombs_path()/sys_wcstombs(). We do
this for several reasons:

- whenever a call site wants to convert a WCS representation of a path or
  file name to an MBS one, it should be made very clear that we *want* the
  special file name conversion to happen.

- it is shorter to read and write.

- future calls to sys_wcstombs() will not incur unwanted conversion by
  accident (it is easy for unsuspecting programmers to assume that the
  function name "sys_wcstombs()" refers to a regular text conversion that
  has nothing to do with paths or filenames).

By keeping the name sys_wcstombs() (and not switching to
sys_wcstombs_path()), the following call sites are implicitly changed to
*exclude* the special path/file name conversion:

cygheap.h (get_drive):
	Cannot contain special characters

external.cc (cygwin_internal):
	Refers to user/domain names, not paths

fhandler_clipboard.cc (fhandler_dev_clipboard::read):
	Is not a path or file name but characters from the Windows
	clipboard

fhandler_console.cc: (dev_console::con_to_str):
	Is not a path or file name but characters from the console

fhandler_registry.cc (encode_regname):
	Is a registry key, not a path or filename

fhandler_registry.cc (multi_wcstombs):
	All call sites pass registry values, not paths or filenames

fhandler_registry.cc (fstat):
	Is a registry value, not a path or filename

fhandler_registry.cc (fill_filebuf):
	Is a registry value, not a path or filename

net.cc (get_ipv4fromreg):
	Is a registry value, not a path or filename

net.cc (get_friendlyname):
	Is a device name, not a path or filename

netdb.cc (open_system_file):
	Is from outside Cygwin

smallprint.cc (__small_vsprintf):
	Is a free text, not a path or filename

strfuncs.cc (strlwr):
	Should preserve the characters from the private page if there
	are any

strfuncs.cc (strupr):
	Should preserve the characters from the private page if there
	are any

uinfo.cc (cygheap_user::init):
	Refers to a user name, not a path or filename

uinfo.cc (pwdgrp::fetch_account_from_windows):
	Refers to value from outside Cygwin

By keeping the function name sys_wcstombs_alloc() (and not changing it to
sys_wcstombs_alloc_path()), the following call sites are implicitly
changed to *exclude* the special path/file name conversion:

ldap.cc (cyg_ldap::remap_uid):
	Refers to a user name, not a path or filename

ldap.cc (cyg_ldap::remap_gid):
	Refers to a group name, not a path or filename

pinfo.cc (_pinfo::cmdline):
	Refers to a command line from Windows, outside Cygwin

uinfo.cc (cygheap_user::env_logsrv):
	Is a server name, not a path or filename

uinfo.cc (cygheap_user::env_domain):
	Refers to the user/domain name, not a path or filename

uinfo.cc (cygheap_user::env_userprofile):
	Refers to Windows' idea of a path, outside Cygwin

uinfo.cc (cygheap_user::env_systemroot):
	Refers to Windows' idea of a path, outside Cygwin

uinfo.cc (fetch_from_description):
	Refers to values from outside of Cygwin

uinfo.cc (cygheap_pwdgrp::get_gecos):
	Refers to user/domain name and email address, not path nor filename

Signed-off-by: Johannes Schindelin <[email protected]>
Windows native symlinks must match the type of their target (file or
directory), otherwise native Windows tools will fail. Creating symlinks in
'nativestrict' mode currently requires the target to exist in order to
check its type.

However, the target of a symlink can change at any time after the symlink
has been created. Thus users of native symlinks must be prepared to deal
with type mismatches anyway. Checking the target type at symlink creation
time is not a good reason to violate the symlink() API specification.

In 'nativestrict' mode, always create native symlinks. Choose the symlink
type according to the target if it exists. Otherwise check the target path
for a trailing '/' as hint to create a directory symlink.

This allows callers to explicitly specify the expected target type, e.g.:

  $ ln -s test/ link-to-test
  $ mkdir test

Signed-off-by: Karsten Blees <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
Internally, Cygwin already uses __utf8_mbtowc(), even if it still claims
to use the "ASCII" charset.

But the `MB_CUR_MAX` value (which is not actually a constant, but
dependent on the current locale) was still 1, which broke the initial
`globify()` call while parsing the the command-line in `build_argv()`
for non-ASCII arguments.

This fixes git-for-windows/git#2189

Signed-off-by: Johannes Schindelin <[email protected]>
This might break things, but it turns out several Windows libraries like
to be loaded at 0x180000000.

This causes a problem, because `msys-2.0.dll` loads at `0x180040000` and
expects `0x180000000-0x180040000` to be available. A problem arises when
Antiviruses (or other DLL hooking mechanisms) load a DLL whose preferred
load address is `0x180000000` and fits in size before `0x180010000`:

1. `msys-2.0.dll` loads and fills `0x180010000-0x180040000` assuming no
   shared console structure is going to be needed.

2. Another DLL loads and fills `0x180000000-0x18000xxxx`

3. `msys-2.0.dll` tries to load `0x180000000-0x180010000` but it's not
   available. It falls back to another address, but down the line
   something else fails.

This bug triggers when using subshells (e.g.: `git clone --recursive`).

The MSYS2 runtime should be able to work around the address conflict,
but the code is failing in some way or other...

Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Mikael Larsson <[email protected]>
Allow overriding the home directory via the HOME variable
Allow native symlinks to non-existing targets in 'nativestrict' mode
This topic branch fixes the problem where a UTF-16 command-line was
converted to UTF-8 in an incorrect way (because Cygwin treated it as if
it was a file name and applied some magic that is intended to allow for
otherwise invalid file names on Windows).

Signed-off-by: Johannes Schindelin <[email protected]>
Workaround certain anti-malware programs

Signed-off-by: Johannes Schindelin <[email protected]>
It frequently leads to problems when trying, say, to call from Git for
Windows' Bash into Cygwin's or MSYS2's, merely because sharing that data
is pretty finicky.

For example, using the Git for Windows that is current at time of
writing, trying to call MSYS2's Bash from Git for Windows' Bash fails
somewhere in `_cmalloc()`, without any error message, and with the
rather misleading exit code 127 (a code which is reserved to indicate
that a command was not found).

Let's just treat these as completely incompatible with one another, by
virtue of using a different `CHILD_INFO_MAGIC` constant.

Let's take the msys2-runtime commit as the tell-tale whether two MSYS2
runtime versions are compatible with each other. To support building in
the MSYS2-packages repository (where we do not check out the
`msys2-runtime` but instead check out Cygwin and apply patches on top),
let's accept a hard-coded commit hash as `./configure` option.

One consequence is that spawned MSYS processes using a different MSYS2
runtime will not be visible as such to the parent process, i.e. they
cannot share any resources such as pseudo terminals. But that's okay,
they are simply treated as if they were regular Win32 programs.

This should also help the scenario where interactions between two
different versions of Git for Windows lead to those infamous `cygheap
base mismatch detected` problems mentioned e.g. in
git-for-windows/git#4255

Note: We have to use a very rare form of encoding the brackets in the
`expr` calls: quadrigraphs (for a thorough explanation, see
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/html_node/Quadrigraphs.html#Quadrigraphs).
This is necessary because it is apparently impossible in `configure.ac`
files to encode brackets otherwise.

Signed-off-by: Johannes Schindelin <[email protected]>
This is a forked repository...

Signed-off-by: Johannes Schindelin <[email protected]>
Having just Cygwin's version in the output of `uname` is not helpful as
both MSYS2 as well as Git for Windows release intermediate versions of
the MSYS2 runtime much more often than Cygwin runtime versions are
released.

Signed-off-by: Johannes Schindelin <[email protected]>
…ys2-runtime-versions-from-sharing-cygheaps

Avoid sharing cygheaps across Cygwin versions
…ys2-runtime-versions-from-sharing-cygheaps-take-two

Prevent different msys2 runtime versions from sharing cygheaps, take two
@dscho dscho self-assigned this May 12, 2023
@dscho dscho marked this pull request as ready for review May 12, 2023 20:30
@dscho
Copy link
Member Author

dscho commented May 12, 2023

I'll just go ahead and merge this. If there are any suggestions how to improve the patches, it is easy enough to do it on top.

@dscho dscho merged commit 3f7f64f into git-for-windows:main May 12, 2023
@dscho dscho deleted the rebase-to-v3.4.6 branch May 12, 2023 20:53
dscho added a commit to git-for-windows/MSYS2-packages that referenced this pull request May 12, 2023
This corresponds to
git-for-windows/msys2-runtime#50.

The `update-patches.sh` script had to be adjusted to reflect that Cygwin
changed its tag naming scheme. Before, it was `cygwin-<version>-release`
where the version had underscores instead of dots. Now, it is
`cygwin-<version>` with dots, not underscores. Also, the source URL
changed to `git+https` but we obviously cannot clone from such a URL.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho added a commit to git-for-windows/MSYS2-packages that referenced this pull request May 12, 2023
This corresponds to
git-for-windows/msys2-runtime#50.

The `update-patches.sh` script had to be adjusted to reflect that Cygwin
changed its tag naming scheme. Before, it was `cygwin-<version>-release`
where the version had underscores instead of dots. Now, it is
`cygwin-<version>` with dots, not underscores. Also, the source URL
changed to `git+https` but we obviously cannot clone from such a URL.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho added a commit to git-for-windows/MSYS2-packages that referenced this pull request May 12, 2023
This corresponds to
git-for-windows/msys2-runtime#50.

The `update-patches.sh` script had to be adjusted to reflect that Cygwin
changed its tag naming scheme. Before, it was `cygwin-<version>-release`
where the version had underscores instead of dots. Now, it is
`cygwin-<version>` with dots, not underscores. Also, the source URL
changed to `git+https` but we obviously cannot clone from such a URL.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho added a commit to git-for-windows/MSYS2-packages that referenced this pull request May 12, 2023
This corresponds to
git-for-windows/msys2-runtime#50.

The `update-patches.sh` script had to be adjusted to reflect that Cygwin
changed its tag naming scheme. Before, it was `cygwin-<version>-release`
where the version had underscores instead of dots. Now, it is
`cygwin-<version>` with dots, not underscores. Also, the source URL
changed to `git+https` but we obviously cannot clone from such a URL.

Signed-off-by: Johannes Schindelin <[email protected]>
dscho added a commit to git-for-windows/MSYS2-packages that referenced this pull request May 13, 2023
This corresponds to
git-for-windows/msys2-runtime#50.

The `update-patches.sh` script had to be adjusted to reflect that Cygwin
changed its tag naming scheme. Before, it was `cygwin-<version>-release`
where the version had underscores instead of dots. Now, it is
`cygwin-<version>` with dots, not underscores. Also, the source URL
changed to `git+https` but we obviously cannot clone from such a URL.

Finally, the build no longer produces an `msys0.dll`, but a
`new-msys-2.0.dll` and the Git for Windows-only code-signing step had to
be adjusted accordingly.

Signed-off-by: Johannes Schindelin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.