Initialization of orthogonal tensors with respect to a pivot #931

pragyasrivastava0805 · 2021-08-25T21:16:57Z

created a new method in tensornetwork/linalg/initialization.py for initializing a random tensor with entries distributed according to normal distribution and performing QR Decomposition on it and returning the tensor Q so that when a tensor is contracted about a given pivot index,the result is orthogonal

google-cla · 2021-08-25T21:17:02Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

pragyasrivastava0805 · 2021-08-26T09:28:23Z

@googlebot I have signed

tensornetwork/backends/abstract_backend.py

mganahl

thanks for the PR! Sorry for the delay, I left some comments for you to fix!

mganahl · 2021-09-07T07:51:25Z

tensornetwork/backends/numpy/numpy_backend.py

+    if ((np.dtype(dtype) is np.dtype(np.complex128)) or
+        (np.dtype(dtype) is np.dtype(np.complex64))):
+      q,r= decompositions.qr(np,np.random.randn(
+          *shape).astype(dtype) + 1j * np.random.randn(*shape).astype(dtype),pivot_axis,non_negative_diagonal)


there is an else clause missing, otherwise line 804 gets overwritten

mganahl · 2021-09-07T07:56:09Z

tensornetwork/linalg/tests/initialization_test.py

+  for dtype in dtypes[backend]["rand"]:
+    tnI = tensornetwork.initialize_orthogonal_tensor_wrt_pivot(
+        shape,
+        dtype=dtype,pivot_axis,


that line should throw a syntax error because your passing an argument between named arguments

mganahl · 2021-09-07T07:56:26Z

tensornetwork/linalg/tests/initialization_test.py

+        shape,
+        dtype=dtype,pivot_axis,
+        seed=seed,
+        backend=backend,non_negative_diagonal)


mganahl · 2021-09-07T08:16:44Z

tensornetwork/linalg/tests/initialization_test.py

+        dtype=dtype,pivot_axis,
+        seed=seed,
+        backend=backend,non_negative_diagonal)
+    npI = backend_obj.initialize_orthogonal_tensor_wrt_pivot(shape, dtype=dtype, pivot_axis, seed=seed,non_negative_diagonal)


remove the function from the backend

mganahl · 2021-09-07T07:58:27Z

tensornetwork/linalg/tests/initialization_test.py

+def test_initialize_orthogonal_tensor_wrt_pivot(backend):
+  shape=(5, 10, 3, 2)
+  pivot_axis=1
+  seed = int(time.time())


pls use deterministic seed initialization

mganahl · 2021-09-07T08:10:45Z

tensornetwork/linalg/initialization.py

@@ -200,3 +201,7 @@ def random_uniform(shape: Sequence[int],
  the_tensor = initialize_tensor("random_uniform", shape, backend=backend,
                                 seed=seed, boundaries=boundaries, dtype=dtype)
  return the_tensor
+def initialize_orthogonal_tensor_wrt_pivot(shape=Sequence[int],dtype:Optional[Type[np.number]]=None,pivot_axis:int=-1,seed=Optional[int]=None,backend: Optional[Union[Text, AbstractBackend]] = None,non_negative_diagonal:bool=False) ->Tensor:
+    the_tensor=initialize_tensor("randn",shape,backend=backend,seed=seed,dtype=dtype)
+    q,r=linalg.qr(the_tensor,pivot_axis,non_negative_diagonal)


us _ instead of r (unused variable)

mganahl · 2021-09-07T08:12:51Z

tensornetwork/linalg/initialization.py

@@ -200,3 +201,7 @@ def random_uniform(shape: Sequence[int],
  the_tensor = initialize_tensor("random_uniform", shape, backend=backend,
                                 seed=seed, boundaries=boundaries, dtype=dtype)
  return the_tensor
+def initialize_orthogonal_tensor_wrt_pivot(shape=Sequence[int],dtype:Optional[Type[np.number]]=None,pivot_axis:int=-1,seed=Optional[int]=None,backend: Optional[Union[Text, AbstractBackend]] = None,non_negative_diagonal:bool=False) ->Tensor:


I'm wondering if we could find a less clunky name. Some possibilities that come to my mind are random_orthogonal or random_isometry @alewis?

Pls add a docstring that explains what the function is doing, what the arguments are, and what the returned values are.

mganahl · 2021-09-07T08:13:58Z

tensornetwork/backends/abstract_backend.py

@@ -1044,3 +1044,6 @@ def eps(self, dtype: Type[np.number]) -> float:



Why did you add this function to the backend? I don't think we need it here

mganahl · 2021-09-07T08:14:44Z

tensornetwork/backends/numpy/numpy_backend.py

@@ -795,3 +795,13 @@ def eps(self, dtype: Type[np.number]) -> float:
      float: Machine epsilon.
    """
    return np.finfo(dtype).eps
+  def initialize_orthogonal_tensor_wrt_pivot(self,shape=Sequence[int],dtype:Optional[Type[np.number]]=None,pivot_axis:int=-1,seed=Optional[int]=None,backend: Optional[Union[Text, AbstractBackend]] = None,non_negative_diagonal: bool = False):->Tensor


I don't think we need this function

mganahl · 2021-09-07T08:17:25Z

tensornetwork/linalg/tests/initialization_test.py

+        seed=seed,
+        backend=backend,non_negative_diagonal)
+    npI = backend_obj.initialize_orthogonal_tensor_wrt_pivot(shape, dtype=dtype, pivot_axis, seed=seed,non_negative_diagonal)
+    np.testing.assert_allclose(tnI.array, npI)


pls replace with a test that checks if the initialized tensor has the desired properties

heading:changed

07f4035

google-cla bot added the cla: no label Aug 25, 2021

pragyasrivastava0805 mentioned this pull request Aug 25, 2021

orthogonalization for tensors #800

Open

google-cla bot added cla: yes and removed cla: no labels Aug 26, 2021

pragyasrivastava0805 added 4 commits September 1, 2021 18:03

Amends Added

a54b55f

Unit-Tests Added

cbb6e1d

Changes

29c730f

Final

55e5d15

mganahl reviewed Sep 6, 2021

View reviewed changes

tensornetwork/backends/abstract_backend.py Show resolved Hide resolved

Breaked the line

c08cd60

mganahl suggested changes Sep 7, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initialization of orthogonal tensors with respect to a pivot #931

Initialization of orthogonal tensors with respect to a pivot #931

pragyasrivastava0805 commented Aug 25, 2021

google-cla bot commented Aug 25, 2021

pragyasrivastava0805 commented Aug 26, 2021

mganahl left a comment

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

mganahl Sep 7, 2021

		@@ -1044,3 +1044,6 @@ def eps(self, dtype: Type[np.number]) -> float:

Initialization of orthogonal tensors with respect to a pivot #931

Are you sure you want to change the base?

Initialization of orthogonal tensors with respect to a pivot #931

Conversation

pragyasrivastava0805 commented Aug 25, 2021

google-cla bot commented Aug 25, 2021

What to do if you already signed the CLA

Individual signers

Corporate signers

pragyasrivastava0805 commented Aug 26, 2021

mganahl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment