I thought deploying the scala fat jar on dreamhost would be easy but I ran into errors loading libraries. I checked and it turns out that dreamhost has java8 installed by default. I uploaded a newer jdk and ran into the following error.
$ jdk-11.0.22+7/bin/java
Error occurred during initialization of VM
Failed to mark memory page as executable - check if grsecurity/PaX is enabDreamhost runs a secure version of linux that refuses to run user uploaded binaries. This is probably a good security decision but makes the service much less useful.
My next tact is to try to get my application to run under java8. It
seems even the latest version of scala 3 should have no problem
running on java8 and sbt even has options to restrict the target and
the jdk. I decided to go with the easy route of just setting
JAVA_HOME and running sbt. Once I did this I still got some
library errors.
The first set of errors involved calls to String.strip() which was
fixed by changing the calls to String.trim(). The next errors were
related to sttp and, after some research, it seems that sttp requires
java 11. I briefly looked at other scala libraries but was worried
that it would be equally hard to track dependencies until after I was
already using them and could see whether or not they actually
worked. So instead I decided to implement the http get with java8 libs
which was long, ugly and painful to get working. Once this was done
everything worked.
After the libraries were fixed I ran the jar build with the new java with no problem on java8. It turns out that it didn’t care about that.
I’m not sure what the take home message of this is. I am both floored that the newest scala can support java8 in some limited fashion. I’m a bit more concerned that my hosting company only supports an ancient version of java and doesn’t support any custom tools. All in all, this is a big plus for scala. If I had a version of my app in a newer version of any other language, I probably would not be able to run it on dreamhost.
I tried to add a dependency for scalatags and ran into a bunch of problems. It seems much of the “getting started” documentation is outdated and you need to go to mavencentral and look for the scala 3 version of the library. After doing that, I’ve got an updated list of dependencies:
libraryDependencies += "org.scala-lang" %% "toolkit" % "0.2.1",
libraryDependencies += "org.scala-lang" %% "toolkit-test" % "0.2.1" % Test,
libraryDependencies += "com.lihaoyi" %% "scalatags" % "0.12.0"I had trouble figuring out how to run my cli program without sbt. The
jar file was generated in <project>/target/scala-3.3.1/*.jar but
when I try to run it with either java -jar or scala I get a bunch
of class not found errors probably due to a missing classpath.
One of the solutions is to make a fat jar that can then be run with java.
Create a new file called project/plugins.sbt and add the following
line to it.
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "2.1.5")The version number is just the latest from the project page: https://github.com/sbt/sbt-assembly
Your build.sbt file needs the following added to the settings:
exportJars := true,Once this is in, you can run
sbt clean
sbt compile
sbt assemblyThis gives you a jar you can throw around and run easily
java -jar ~/stock-notes-sc/target/scala-3.3.1/stock-notes-sc-assembly-0.1.0-SNAPSHOT.jar oldestNow that I’m retired, I’m getting into Scala a bit. The new way to
install scala is to run a curl script that downloads and runs
something called coursier which ends up installing a bunch of scala
development programs into .local/share/coursier/bin the most
important are probably sbt, amm, and scala-cli.
I’ve also downloaded VSCode and metals which is supposed to give me a decent scala development experience.
The command to create a new scala3 command line project is:
sbt new scala/scala3.g8
#enter the github passphrase
#enter the name of the project; a new directory in the current dir
# will be made with that name
sbt runA lot of people hate SBT but it is so standard it is easy to find documentation on it. I only expect to have dirt simple builds so it should be fine. I might switch to mill when that becomes more standard.
If you open the new project directory in a metal enabled VSCode it
appears to build, run, debug, and navigation the program
correctly. Opening a file named hello.worksheet.sc gives you a scala
worksheet than can dynamically evaluate scala commands.
In the SBT file it’s useful to get rid of the munit library and replace it with the Scala Toolkit. This gives you munit, os-lib, uPickle (JSON), and sttp (an HTTP library)
//libraryDependencies += "org.scalameta" %% "munit" % "0.7.29" % Test
libraryDependencies += "org.scala-lang" %% "toolkit" % "0.1.7",
libraryDependencies += "org.scala-lang" %% "toolkit-test" % "0.1.7" % TestI think my first project will be to write a near drop-in replacement for my stock quote program. I’ll make a project and start drafting requirements.
I haven’t had a need for Scala in a couple years.
My friend Tony sent me a free ebook “Functional, Programming Simplified: (Scala Edition).” It had enough good points that it’s worth a little note taking.
The author of the book grew up with clasical imperative / OOP methodologies and has since been using Scala to learn and implement projects with functional methodologies. His background gives him a good voice to folks who haven’t already drunk the functional kool-aide.
Each chapter is short and focused and fenced by a Goal and Review section. He starts the book saying “if you already know a concept skip the chapter”. This was good advice and the focused nature of the chapters made it easy to know if the topic was skip-able.
There is a lot of talk about the benefits of immutable values and pure functions especially as they apply to concurrency, clarity and technical debt. I became a fan of this decades ago and had no idea I was adopting some aspect of functional programming. The book goes into a lot of specifics on the benefits of immutability and pure functions but, for me, it was preaching to the choir.
The de-mystification of the FP lingo was very refreshing
| FP | English |
|---|---|
| Lambda | Anoymous Function |
| map | creates a destination object with transformed elements of the source object |
| flatMap | like map but each source element can be transformed into zero or more destination elements |
| Functor | an object than implements map |
| Monad | An object that implements map and flatMap *1 |
| Future | Concurrent Process *2 |
- *1
- In theory a Monad must also have a point/lift/apply ability but that doesn’t really matter in Scala. There are also Applicatives but these don’t matter to Scala either.
- *2
- I think of Futures as “processes who’s result may or may not have completed” but I agree that the name is terminology that needs to be learned.
Really the only thing that matters to Scala is that if an object implements map and flatMap then it can be used in a for expression. That’s it! Also, “For something to be used in a for expression, it doesn’t have to be a class that implements map and flatMap methods; it just need to return a type that implements map and flatMap methods”
For that matter, did it help for Scala to call them “for expressions”? Does this just confuse imperative programmers in thinking they are looping when they are just calling generators and map functions? Maybe a better name is “mapping expression” or “mapping comprehension”.
There was a lot of honesty that you don’t hear from the FP camp. One example: “If an FP advocate says ‘The IO monad makes a function that handles IO pure’ they are lying.” I wasted a lot of time trying to understand how IO could be handled in pure functions. If someone had told me this early in my functional learning it would have saved me a bunch of time.
I really like the concept of an IO tag as a way to document impure functions. It’s a shame that it doesn’t work as the author exhaustively shows. Basically, having two levels of Monad such as Try[IO[String]] makes them difficult to unwrap in a for expression. Some frameworks work around this by making separate monads: FileIO, NetworkIO and DatabaseIO
This brings me to the other standard monads. I love functional error handling with: Option, Try, Or, Either. These are genious.
Domain modeling OOP vs FP was interesting. It shows the power of FP for modeling very complex problems that have to be divided among multiple groups. I don’t think he did a good job of selling the pure functional architecture. I think he believes this is the best architecture but it didn’t do it for me. Maybe I don’t work on any projects that are large enough to benefit from that kind of separation?
The functional objects approach was an easy sell to me. I love the builder pattern:
val p = Pizza().addTopping(Pepperoni()).addTopping(Mushroom())The fact that the scala collection classes follow this pattern makes me think it’s extra “blessed”
Case class copy and Lenses look like a great way to copy/update immutable objects. I don’t remember this from the Odersky classes.
Futures are simple and awesome. They’re a great way to replace any system that uses callbacks. They really leverage the scala “call by name” language feature.
Scala is so powerful you can write your own control groups (whilst). I liked his example.
It bugged me that Random class was used in a functional example was not immutable. He mutated it and passed the reference around as if it was immutable.
It bugs me that Scala has all of these other fundemental libraries: Cats, Scalaz. Will one of these ever be blessed and incorporated into the standard library?
Property based testing (ScalaCheck) should interest me more than it does. I haven’t tried it but I have several guys on my team that have tried it and they are not that impressed. The concensus is that it’s not a bad way to find the optimal unit or functional test but it generally increases testing time a huge amount with dubious tests. I’d still like to try them.
There were early discussions that creating composable functionality is superior to temporary values. e.g.
z=f(g(x))is better than
y=g(x)
z=f(y)…this doesn’t make any sense to me. Always use temp values when they improve readability and remove them when they obscure readability.
There are a whole class of mathematical gymnastics that FP folks like to engage in just to allow a for expression to work with multiple types of mondads. This is summed up in the whole section on Monad Transformers. For the life of me I could not see how this was useful. Are nested for expressions so bad?
There was also a lot of talke about functional state. State can be as simple as “take an immutable argument and return an immutable value of the same type” Why make it more complicated than that?
I haven’t looked at Scala in a while and was reminded how punctuation heavy it is. I don’t think Odersky ever met a punctuation character he didn’t like: foldLeft, FoldRight, pass by value, trait requires. Can you remember the punctuation that performs these concepts? This unreadability may explain the rise of Kotlin in spite of Scala being such a richer language.
None of the authors arguments on partially applied functions and currying sold me on their value.
He only had a couple chapters on Akka and actors so maybe I shouldn’t ding him on this but I still haven’t seen a good actor example that makes me say “this is a great way to solve this problem”. How do actors mix with microservices if at all? Can actors be mixed between services or does the actor “system” have to be a monolithic application?
Parallel collections are easy but don’t seem to have an immediate use. I would always choose to handle concurrency at a higher level.
Minor success. Added a junit test to the android-sbt-plugin autogenerated project and was able to run it via “sbt android:test”
Tracking changes in gen-android branch custom1
The scaloid sample app seems to have a strange directory structure
src/scaloid/example/HelloScaloid.scalaAndroid usually puts tests in a separate project. I wonder if that’s how this works?
Added android-sdk-plugin to global plugins:
~/.sbt/0.13/plugins/build.sbt
addSbtPlugin("com.hanhuy.sbt" % "android-sdk-plugin" % "1.3.10")This allows generation of new android projects via:
mkdir test1
cd test1
sbt "gen-android android-19 org.kleemann.test1 test1"Strange that it produces all java files. What’s the point?
Having lots of problems getting tests to run. The above android-sdk-plugin call generates the following directory structure.
src/
src/androidTest
src/androidTest/java
src/androidTest/java/org
src/androidTest/java/org/kleemann
src/androidTest/java/org/kleemann/MainActivityTest.java
src/main
src/main/AndroidManifest.xml
src/main/java
src/main/java/org
src/main/java/org/kleemann
src/main/java/org/kleemann/MainActivity.java
src/main/res
src/main/res/drawable-xhdpi
src/main/res/drawable-xhdpi/ic_launcher.png
src/main/res/drawable-mdpi
src/main/res/drawable-mdpi/ic_launcher.png
src/main/res/drawable-ldpi
src/main/res/drawable-ldpi/ic_launcher.png
src/main/res/drawable-hdpi
src/main/res/drawable-hdpi/ic_launcher.png
src/main/res/layout
src/main/res/layout/main.xml
src/main/res/values
src/main/res/values/strings.xmlThere seems to be two possible ways to configure the sbt build:
build.sbt
project/
build.scala
build.properties
project/
plugins.sbt
build.propertiesFailure to find zipalign requires a new version of android-sdk-plugin Change via project/plugins.sbt from 1.3.6 to 1.3.10
“The SDK Build Tools revision (19.0.3) is too low for project ‘hello-scaloid-sbt’. Minimum required is 19.1.0” Updated via “android sdk” Finally compiled. “android avd&” and “sbt run” runs the app.
“sbt ~run” allows dynamic rebuilds and redeployment. Editing files via emacs. Maybe I should try to do this with sbt, emacs, and the command line tools for a while. Will give me some nuts and bolts experience without an IDE. Starting with the old, moldy scala mode that comes with Ubuntu: scala-dist/tool-support/src/emacs The newer version hvesalai/scala-mode2 would have to be installed manually.
I’m going to be trying some Android development in scala. This is going to require:
- The android SDK (not necessarily the whole ADT custom eclipse) just ANDROID_HOME pointing to the sdk directory
- the android-sbt-plugin for sbt seems to be needed to make easy sbt builds. This handles the complicated stuff such as resource building and proguard post-processing to cut the jar down to a managable size.
- scaloid is a library that adds scala like wrappers around common android calls. This seems necessary.
- For eclipse it seems the best thing to do is install the scala ide and then add the android plugins to it. Some icons seem missing as apposed to the eclipse android bundle.
Unfortunately all of this isn’t a happy place.
- The android-sbt and scaloid folks want the primary build to be an sbt project and then have your IDE plug into that. They seem to like IDEA and hate Eclipse so the Eclipse support sucks.
- The Scala IDE (Eclipse) folks don’t seem to care much about Android and even suggest starting an Eclipse android java project and adding scala to it after the fact. This would prevent external builds.
- Google seems to be transitioning from Eclipse as the official Android IDE to IDEA.
I’m not sure where to go with all of this. I am used to Eclipse and have nothing against IDEA but I’d rather not have to start paying for an editor.
Scala IDE 3.0.4; Scala 2.10
Hello scaloid github project currently needs sbt 0.13.5 or above, needs Android SDK level 10
Installing sbt v0.13.7 to give it a shot instead of activator
Ran into a case where I was writing a pure function that reformats a string for pretty printing. The natural solution that came to me was imperative but I thought I would implement it a functional way just to compare them. I like the imperative solution better.
/**
* Add some newlines and indentation to make the objects easier to read
*/
def prettyPrint: String = {
// it would be interesting to see this functionally instead of imperatively
// I would imagine lots of objects would be generated in order to handle indents
var indent = 0
val sb = new StringBuilder()
for (c <- toString) {
c match {
case '(' => {
sb.append("(\n")
indent += 2
sb.append(" " * indent)
}
case ')' => {
sb.append(")\n")
indent -= 2
sb.append(" " * indent)
}
case ',' => {
sb.append(",\n")
sb.append(" " * indent)
}
case _ => sb.append(c)
}
}
sb.toString
}// this is shorter but way more obscure than the imperative approach
def prettyPrintFunctional: String = {
toString.foldLeft((0, "")){ (t, c) =>
t match {
case (i, s) => c match {
case '(' => (i+2, s + "(\n" + " " * (i+2))
case ')' => (i-2, s + ")\n" + " " * (i-2))
case ',' => (i, s + ")\n" + " " * i)
case _ => (i, s + c)
}
}
}._2
}I think functional solutions run into problems when you have to pass a large amount of state between iterations.
Trying to do a couple things:
- use case classes with matching instead of heavyweight OOP classes
- favor functional style and immutablilty
This is probably not the best choice in all cases but it gives me a little more experience with these other programming paradigms
For some reason I can run the eclipsify command from the command line now:
$ activator eclipse…I swear this wasn’t working yesterday. I think with a fresh install of the activator, you may have to run the “make eclipse files” once with the ui before you are able to run it from the command line.
Not sure if all the files should be added to the project. Adding the activator jar seems strange. Stackoverflow says their only purpose is for allowing builds for people that don’t have activator installed. Seems kind of dumb. My policy will be to delete them.
Maybe I should keep that new project snippet in a separate place. If I use it a lot, it may make sense to make my own template or bash script.
TODO: learn more about scala-ide, formatting, etc.
Selecting a source file or package from the package explorer allows the “source” menu to appear which allows the “format” command. This is a good way to clean up poorly formatted scala. Seems to also get rid of hard tabs.
TODO: check for tabs in source code
It seems scala formatting inherits from java. Need to make a custom java formatter that doesn’t use tabs.
https://www.assembla.com/spaces/scala-ide/tickets/1000198#/activity/ticket:
TODO: see if Scalastyle is used by the ide
TODO: the parsing shown in scells is more interesting than I had thought. Read the chapter on combinator parsing.
TODO: configure eclipse to trim trailing spaces
I probably should have been doing this a while ago. Now is better than nothing.
While reading functional programming with scala I was working with a lot of functions that were returning functions. It turns out that that syntax produces an identical signature as the curry syntax. e.g.
object curry {
def f1(a: Int): Int => Int =
b => a + b //> f1: (a: Int)Int => Int
def f2(a: Int)(b: Int): Int =
a + b //> f2: (a: Int)(b: Int)Int
// note: both functions can be assigned to x and y which means they have the same type
var x = f1(42) //> x : Int => Int = <function1>
x(3) //> res0: Int = 45
x = f2(42)
x(3) //> res1: Int = 45
var y = f2 _ //> y : Int => (Int => Int) = <function1>
y(42)(3) //> res2: Int = 45
y = f1 _
y(42)(3) //> res3: Int = 45
}Try to learn the basics of sbt, activator, eclipse. Some minimal ways to get started:
Had to add sbteclipse to plugins dir
cd ~/.sbt
mkdir plugins
echo 'addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "2.1.2")' >> ~/.sbt/plugins/plugins.sbtThe above command did not seem to allow eclipse to be run from activator. I was only able to create the eclipse project files with “activator ui”
activator new myproject minimal-scala
cd myproject
rm activator*
git init
git status
<add .cache .classpath .project to .gitignore>
git add <files listed in status>
git commit -a -m "initial changes to template"
<add project via github website>
git remote add origin https://github.com/sizezero/PROJECTNAME.git
git push origin master
activator compile
activator eclipse