Thursday, August 17, 2017

Julia - Language - Functions - Single Expression Functions

These functions can be written in a single expression and mimic the way we solve maths functions on a piece of paper. Consider the popular function, f(x) = x*x = x^2


$ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0 (2017-06-19 13:05 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> f(x) = x^2
f (generic function with 1 method)

julia> f(2)
4


Let us see all the methods which we can pass to our square function.

julia> methods(f)
# 1 method for generic function "f":
f(x) in Main at REPL[1]:1

julia> 

We can see that, single expression function is very convenient. It is a shorthand and almost a mathematical syntax. A single expression function is a generic function with no name.

Tuesday, August 15, 2017

Julia - Language - Collection - Dictionary

Dictionaries are part of standard Julia Language collections. Information are stored in it in form of key-value pairs. We can think of them as arrays in which indices are keys with corresponding value. 

Standard Julia Array:


array value 45 65 20 8
array index 1 2 3 4




Standard Julia Dictionary:



dictionary value Panjim Hyderabad Bengaluru Chandigarh
dictionary key Goa Telangana Karnataka Punjab



This dictionary can be created using the following syntax:

Technique 1: Using => shorthand

julia> dict_country_capitals1 = Dict("Goa" => "Panjim", "Telangana" => "Hyderabad", "Karnataka" => "Bengaluru", "Punjab" => "Chandigarh")
Dict{String,String} with 4 entries:
  "Punjab"    => "Chandigarh"
  "Telangana" => "Hyderabad"
  "Karnataka" => "Bengaluru"
  "Goa"       => "Panjim"


julia>

The symbol => is a shorthand for the Pair function. 

Technique 2: Using Pair function

The second technique is to use the Pair Function to create this key-value pair:

julia> dict_country_capitals2 = Dict(Pair("Goa" , "Panjim") , Pair("Telangana" , "Hyderabad") , Pair("Karnataka" , "Bengaluru") , Pair("Punjab" , "Chandigarh"))
Dict{String,String} with 4 entries:
  "Punjab"    => "Chandigarh"
  "Telangana" => "Hyderabad"
  "Karnataka" => "Bengaluru"
  "Goa"       => "Panjim"


julia> 

To access the value in a Dictionary, we must pass the key as a input parameter to the Dictionary:

julia> dict_country_capitals1["Goa"]
"Panjim"

julia> 

Technique 3: Dynamic Dictionary - Key-Value are any datatype
We can also create Dynamic Dictionary in which any datatype can be a Key and Value.

julia> dict3 = Dict{Any,Any}(1 => "I dont know!!","Goa" => "Panjim", "Telangana" => "Hyderabad", "Karnataka" => "Bengaluru", "Punjab" => "Chandigarh")
Dict{Any,Any} with 5 entries:
  "Punjab"    => "Chandigarh"
  "Telangana" => "Hyderabad"
  "Karnataka" => "Bengaluru"
  "Goa"       => "Panjim"
  1           => "I dont know!!"

julia> dict3
Dict{Any,Any} with 5 entries:
  "Punjab"    => "Chandigarh"
  "Telangana" => "Hyderabad"
  "Karnataka" => "Bengaluru"
  "Goa"       => "Panjim"
  1           => "I dont know!!"

Accessing the Dictionary Values using Keys:

julia> dict3[1]
"I dont know!!"

julia> dict3["Goa"]
"Panjim"

julia> 


Technique 4: Symbol Notation

julia> dict_symbol_notation = Dict(:A => "America", :B => "Asia", :C => "Australia")
Dict{Symbol,String} with 3 entries:
  :A => "America"
  :B => "Asia"
  :C => "Australia"

Accessing the Dictionary Values using Keys:

julia> dict_symbol_notation[:B]
"Asia"

julia> 


Handling Exceptions: In case a key does not exist in a dictionary, and we try to access it - then Julia will throw an error. To avoid an error, we can use the get() function at the time of accessing:

Julia throws error, if the key is not found. 

julia> dict_symbol_notation[:H]
ERROR: KeyError: key :H not found
Stacktrace:
 [1] getindex(::Dict{Symbol,String}, ::Symbol) at ./dict.jl:474

using get() function avoids Julia Error:

julia> get(dict_symbol_notation,:H,"Key not found!!")
"Key not found!!"

julia> 

in() function: to check if a key value pair exists in a dictionary:

julia> in((:A => "America"),dict_symbol_notation)
true

haskey() function: to check if a key exists in a dictionary:

julia> haskey(dict_symbol_notation,:A)
true

julia> haskey(dict_symbol_notation,:H)
false

Since, H is not existing, to add a key-value pair to a dictionary is very easy. We will use the assignment operator.

julia> dict_symbol_notation[:H] = "Africa"
"Africa"

julia> dict_symbol_notation
Dict{Symbol,String} with 4 entries:
  :A => "America"
  :H => "Africa"
  :B => "Asia"
  :C => "Australia"

julia> 

Therefore, dictionaries are mutable. That means, that their values can be changed. Let us edit the value for the symbol, :C

julia> dict_symbol_notation[:C] = "Europe"
"Europe"

julia> dict_symbol_notation
Dict{Symbol,String} with 4 entries:
  :A => "America"
  :H => "Africa"
  :B => "Asia"
  :C => "Europe"

julia> 

delete!() function: to delete a entry from dictionary:

julia> delete!(dict_symbol_notation,:C)
Dict{Symbol,String} with 3 entries:
  :A => "America"
  :H => "Africa"
  :B => "Asia"

julia> 

length() function: to know the number of key/values in a dictionary:

julia> length(dict_symbol_notation)
3

A dictionary has no size() and no ndims()

julia> size(dict_symbol_notation)
ERROR: MethodError: no method matching size(::Dict{Symbol,String})
Closest candidates are:
  size(::Any, ::Integer, ::Integer, ::Integer...) where N at abstractarray.jl:30
  size(::Char) at char.jl:13
  size(::Char, ::Any) at char.jl:14
  ...

julia> 

A for loop can be used to read and populate a dictionary:

julia> for (k,v) in dict_symbol_notation
       println("The key $(k) has a value $(v)")
       end
The key A has a value America
The key H has a value Africa
The key B has a value Asia

julia> 


To convert a standard Julia Array to Dictionary, Let us consider a standard Julia array of Strings:

julia> procedure_values = ["Cardiology","Physiology","Microbiology"]
3-element Array{String,1}:
 "Cardiology"  
 "Physiology"  
 "Microbiology"

Now, let us create a dictionary from this:

julia> procedure_dict = Dict{AbstractString,AbstractString}()
Dict{AbstractString,AbstractString} with 0 entries

julia> procedure_dict
Dict{AbstractString,AbstractString} with 0 entries

julia> enumerate(procedure_values)
Enumerate{Array{String,1}}(String["Cardiology", "Physiology", "Microbiology"])

Use for loop and enumerate function to fill the dictionary:

julia> for (s,n) in enumerate(procedure_values)
       procedure_dict["x_$(s)"] = n
       end

Check the Keys in the new dictionary using the keys() function:

julia> keys(procedure_dict)
Base.KeyIterator for a Dict{AbstractString,AbstractString} with 3 entries. Keys:
  "x_1"
  "x_2"
  "x_3"


Sorting a dictionary output:

Let us create a dictionary. 

julia> dict4Sort = Dict{AbstractString,Int16}("a" => 1, "b" => 2, "c" => 3, "d" => 4, "e" => 5, "f" => 6)
Dict{AbstractString,Int16} with 6 entries:
  "f" => 6
  "c" => 3
  "e" => 5
  "b" => 2
  "a" => 1
  "d" => 4

julia>

We can see that the output is not in sorted format. To sort them, we need to use the sort function available in Julia.

First, let us list the keys in the dictionary using the keys() function:

julia> keys(dict4Sort)
Base.KeyIterator for a Dict{AbstractString,Int16} with 6 entries. Keys:
  "f"
  "c"
  "e"
  "b"
  "a"
  "d"

Let us now use the collect function to gather the keys into an array:

julia> collect(keys(dict4Sort))
6-element Array{AbstractString,1}:
 "f"
 "c"
 "e"
 "b"
 "a"
 "d"

We can use the sort() function to provide a sorted output:

julia> sort(collect(keys(dict4Sort)))
6-element Array{AbstractString,1}:
 "a"
 "b"
 "c"
 "d"
 "e"
 "f"

julia> 

Using a for loop we can print the values:

julia> for i in sort(collect(keys(dict4Sort)))
       println("$(i) => $(dict4Sort[i])")
       end
a => 1
b => 2
c => 3
d => 4
e => 5
f => 6


julia> 


Monday, August 14, 2017

Julia - Language - Install Juno Text Editor

To Install Juno text editor, first install Atom text editor, created by GitHub. Download the Debian package of Atom from the link: https://atom.io/

Download Atom text editor:

$ wget https://atom-installer.github.com/v1.19.2/atom-amd64.deb
--2017-08-21 17:26:25--  https://atom-installer.github.com/v1.19.2/atom-amd64.deb
Resolving atom-installer.github.com (atom-installer.github.com)... 151.101.0.133, 151.101.64.133, 151.101.128.133, ...
Connecting to atom-installer.github.com (atom-installer.github.com)|151.101.0.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83638506 (80M) [application/octet-stream]
Saving to: ‘atom-amd64.deb’

atom-amd64.deb                                     100%[================================================================================================================>]  79.76M  1.87MB/s    in 43s     

2017-08-21 17:27:10 (1.85 MB/s) - ‘atom-amd64.deb’ saved [83638506/83638506]

$

Now install using the dpkg utility of debian / ubuntu / linux mint:

$ sudo dpkg -i atom-amd64.deb 
Selecting previously unselected package atom.
(Reading database ... 174639 files and directories currently installed.)
Preparing to unpack atom-amd64.deb ...
Unpacking atom (1.19.0) ...
dpkg: dependency problems prevent configuration of atom:
 atom depends on git; however:
  Package git is not installed.

dpkg: error processing package atom (--install):
 dependency problems - leaving unconfigured
Processing triggers for gnome-menus (3.10.1-0ubuntu2) ...
Processing triggers for desktop-file-utils (0.22-1ubuntu1) ...
Processing triggers for mime-support (3.54ubuntu1.1) ...
Errors were encountered while processing:
 atom

We can see that there were some errors due to missing dependencies. Hence to resolve them, let us run:

 $ sudo apt-get install -f

This will install atom. To confirm, let us run the following commands:

$ which atom
/usr/bin/atom

$ atom --version
Atom    : 1.19.0
Electron: 1.6.9
Chrome  : 56.0.2924.87
Node    : 7.4.0


In Atom, go to Settings (Ctrl ,, or Cmd , on OS X) and go to the "Install" panel.

Type uber-juno into the search box and hit enter. Click the install button on the package of the same name.

Atom will then set up Juno for you, installing the required Atom and Julia packages. Once the setup is done you're good to go!





Julia - Language - function information and method signatures

In Julia Language, to find out basic method information of a function, simply type the function name at the Julia prompt:

julia> println
println (generic function with 3 methods)


To find out the signatures of those methods, we use the methods() function:

julia> methods(println)
# 3 methods for generic function "println":
println(io::IO) in Base at coreio.jl:6
println(io::IO, xs...) in Base at strings/io.jl:54
println(xs...) in Base at coreio.jl:5

julia>

Refer: https://docs.julialang.org/en/stable/manual/methods/

Julia - Language - DataFrames - Spreadsheet like data structures

In real world, multiple datasets have relations between them. Hence, information needs to be represented in  formats like : Spreadsheets or Tables. We cannot hope to address this issue using DataArrays. To solve this problem of storing relations between data in a single data structure we can use DataFrames in Julia.

Corresponding package to DataFrames in R and Python:
Julia
Python
R
DataFrames
Pandas
data.frame

Consider the below example:

$ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0 (2017-06-19 13:05 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> using DataFrames
INFO: Precompiling module DataFrames.

julia> waugh_df = DataFrame(Name=["Steve Waugh","Mark Waugh"], Age = [52,52], Country = ["Australia","Australia"])
2×3 DataFrames.DataFrame
│ Row │ Name          │ Age │ Country     │
├─────┼───────────────┼─────┼─────────────┤
│ 1   │ "Steve Waugh" │ 52  │ "Australia" │
│ 2   │ "Mark Waugh"  │ 52  │ "Australia" │

julia> 


Now, we cannot create the above dataset using DataArrays. This dataset has the following features:
  1. Labelling of Columns
  2. Different Datatypes in different columns. Remember that matrix can only contain value of one type.
  3. Data Elements have relation with other data elements in the same row of different columns. See, we may argue, I can use multiple vectors and maintain relations between them. But, Vectors cannot be enforced to have the same length. 

Note: A column in DataFrame is represented by a DataArray. 


Technique 2:

Create the DataFrame by calling the DataFrame constructor:
julia> cricket_score_df = DataFrame()
0×0 DataFrames.DataFrame

Add one column, say Players:

julia> cricket_score_df[:Player] = ["Adam Gilchrist", "Mahendra Singh Dhoni","Jonty Rhodes","Sanath Jayasurya","Arjuna Ranatunga","Wasim Akram","Bret Lee","Brian Lara","Lance Klusner","Yuvaraj Singh"]
10-element Array{String,1}:
 "Adam Gilchrist"      
 "Mahendra Singh Dhoni"
 "Jonty Rhodes"        
 "Sanath Jayasurya"    
 "Arjuna Ranatunga"    
 "Wasim Akram"         
 "Bret Lee"            
 "Brian Lara"          
 "Lance Klusner"       
 "Yuvaraj Singh"       

Add second column, say, Score:

julia> cricket_score_df[:Score] = [100,101,102,103,104,105,106,107,108,109]
10-element Array{Int64,1}:
 100
 101
 102
 103
 104
 105
 106
 107
 108
 109

Now, let us check the contents of the DataFrame:

julia> cricket_score_df
10×2 DataFrames.DataFrame
│ Row │ Player                 │ Score │
├─────┼────────────────────────┼───────┤
│ 1   │ "Adam Gilchrist"       │ 100   │
│ 2   │ "Mahendra Singh Dhoni" │ 101   │
│ 3   │ "Jonty Rhodes"         │ 102   │
│ 4   │ "Sanath Jayasurya"     │ 103   │
│ 5   │ "Arjuna Ranatunga"     │ 104   │
│ 6   │ "Wasim Akram"          │ 105   │
│ 7   │ "Bret Lee"             │ 106   │
│ 8   │ "Brian Lara"           │ 107   │
│ 9   │ "Lance Klusner"        │ 108   │
│ 10  │ "Yuvaraj Singh"        │ 109   │

julia> 

We have created a DataFrame. 

Finding Size of a DataFrame:

julia> size(cricket_score_df)
(10, 2)

Use head and tail functions to observe the first few lines of a DataFrame and last few lines respectively:

julia> head(cricket_score_df)
6×2 DataFrames.DataFrame
│ Row │ Player                 │ Score │
├─────┼────────────────────────┼───────┤
│ 1   │ "Adam Gilchrist"       │ 100   │
│ 2   │ "Mahendra Singh Dhoni" │ 101   │
│ 3   │ "Jonty Rhodes"         │ 102   │
│ 4   │ "Sanath Jayasurya"     │ 103   │
│ 5   │ "Arjuna Ranatunga"     │ 104   │
│ 6   │ "Wasim Akram"          │ 105   │

julia> tail(cricket_score_df)
6×2 DataFrames.DataFrame
│ Row │ Player             │ Score │
├─────┼────────────────────┼───────┤
│ 1   │ "Arjuna Ranatunga" │ 104   │
│ 2   │ "Wasim Akram"      │ 105   │
│ 3   │ "Bret Lee"         │ 106   │
│ 4   │ "Brian Lara"       │ 107   │
│ 5   │ "Lance Klusner"    │ 108   │
│ 6   │ "Yuvaraj Singh"    │ 109   │


To access the data in a particular column:

julia> cricket_score_df[:Player]
10-element DataArrays.DataArray{String,1}:
 "Adam Gilchrist"      
 "Mahendra Singh Dhoni"
 "Jonty Rhodes"        
 "Sanath Jayasurya"    
 "Arjuna Ranatunga"    
 "Wasim Akram"         
 "Bret Lee"            
 "Brian Lara"          
 "Lance Klusner"       
 "Yuvaraj Singh"       

julia> 

By providing real world names to the columns, we do not have the need anymore to remember the numeric indices of these columns as they can be accessed by their real world names. 

We can rename the columns by using the rename!() function:


julia> rename!(cricket_score_df,:Player,:TeamPlayer)
10×2 DataFrames.DataFrame
│ Row │ TeamPlayer             │ Score │
├─────┼────────────────────────┼───────┤
│ 1   │ "Adam Gilchrist"       │ 100   │
│ 2   │ "Mahendra Singh Dhoni" │ 101   │
│ 3   │ "Jonty Rhodes"         │ 102   │
│ 4   │ "Sanath Jayasurya"     │ 103   │
│ 5   │ "Arjuna Ranatunga"     │ 104   │
│ 6   │ "Wasim Akram"          │ 105   │
│ 7   │ "Bret Lee"             │ 106   │
│ 8   │ "Brian Lara"           │ 107   │
│ 9   │ "Lance Klusner"        │ 108   │
│ 10  │ "Yuvaraj Singh"        │ 109   │

julia> 


Using the describe function, Julia summarizes the dataset for us:

julia> describe(cricket_score_df)
TeamPlayer
Summary Stats:
Length:         10
Type:           String
Number Unique:  10
Number Missing: 0
% Missing:      0.000000

Score
Summary Stats:
Mean:           104.500000
Minimum:        100.000000
1st Quartile:   102.250000
Median:         104.500000
3rd Quartile:   106.750000
Maximum:        109.000000
Length:         10
Type:           Int64
Number Missing: 0
% Missing:      0.000000



To List the column names, use the names() function:

julia> names(cricket_score_df)
2-element Array{Symbol,1}:
 :TeamPlayer
 :Score     





Friday, August 11, 2017

Julia - Language - DataArrays - Vectors and Matrices

The advantage of using DataArrays over standard Array type in Julia is that we can store values with "no value" (NA). Also, similar to standard array type in Julia we can create:

  • vectors
  • matrices

Vectors - One Dimensional Array:


To create a vector using standard array:

julia> array_1d = [1,2,3,4,5]
5-element Array{Int64,1}:
 1

 2
 3
 4
 5

To create a data vector using dataarray's:


julia> darray_1d = data([1,2,3,4,5])
5-element DataArrays.DataArray{Int64,1}:
 1
 2
 3
 4
 5


Matrix - Two Dimensional Array:

Creating a 2 dimensional matrix in standard Julia:

julia> matrix_2d = [1 2 3 ; 4 5 6]
2x3 Array{Int64,2}:
 1  2  3
 4  5  6

Creating a 2 dimensional matrix using Data Arrays.

julia> da_matrix_2d = data([1 2 3 ; 4 5 6])
2x3 DataArrays.DataArray{Int64,2}:
 1  2  3
 4  5  6


We see that the main advantage of using DataArrays in Julia over standard arrays is that we can use NA values ("no value", NULL, missing value) in DataArrays. But, dataarrays are having one main limitation - we cannot use it to store multiple datatypes in multiple columns. Basically, we cannot use it like an excel sheet.

Now, to overcome this limitation of dataarrays, julia recommends us to utilize the DataFrames package.

Julia - Language - NA datatype and DataArrays Package

In Julia, consider an array "a" having 4 values:


$ julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: https://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.6.0 (2017-06-19 13:05 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-pc-linux-gnu

julia> a = [1,2,3,4]
4-element Array{Int64,1}:
 1
 2
 3
 4


Consider the case of missing values. Now, If we want to set the element 1 in array a to "no value". How would you do that? Let us try the following:

julia> a[1] = ""
ERROR: MethodError: Cannot `convert` an object of type String to an object of type Int64
This may have arisen from a call to the constructor Int64(...),
since type constructors fall back to convert methods.
Stacktrace:
 [1] setindex!(::Array{Int64,1}, ::String, ::Int64) at ./array.jl:549

julia> a[1] =
       ;

ERROR: syntax: unexpected ;

julia>
We find that, it is not possible to assign "no value" to elements in regular Julia arrays. Hence, to to represent this concept of "no value" or missing values, Julia provides this singleton object, NA. Now, let us try to use NA on regular Julia arrays:

julia> a[1] = NA
ERROR: UndefVarError: NA not defined

We find that NA is not available to use directly. To be able to use NA, we load the DataArrays package:

julia> using DataArrays

julia> a[1] = NA
ERROR: MethodError: Cannot `convert` an object of type DataArrays.NAtype to an object of type Int64
This may have arisen from a call to the constructor Int64(...),
since type constructors fall back to convert methods.
Stacktrace:
 [1] setindex!(::Array{Int64,1}, ::DataArrays.NAtype, ::Int64) at ./array.jl:549

When we use NA ("no value / not available / null / missing value ") on regular Julia arrays, it is not allowing us to include it as an element. Hence, we need to use DataArray instead of regular Julia arrays:

julia> b = DataArray([1,2,3,4])
4-element DataArrays.DataArray{Int64,1}:
 1
 2
 3
 4

Now, let us assign the value NA.

julia> b[1] = NA
NA

julia> b
4-element DataArrays.DataArray{Int64,1}:
  NA
 2 
 3 
 4 

julia>

Success!

Now, Let us try to calculate mean over the elements of vector "b":

julia> mean(b)
NA


We see that Julia does not allow us to compute mean over a vector array having NA element. Hence, we compute mean by slicing the vector array "b":

julia> mean(b[2:end])
3.0


But, using this technique is very inconvenient, as - to be able to slice a vector array - it expects us to know in advance which elements are having NA values. It is better to use dropna function:

julia> mean(dropna(b))
3.0

We can see that dropna function has enabled us to overcome this drawback of slicing and also the mean is calculated over the remaining 3 elements.