Posted in Information Technology

StackOverflow: Seven of the Best Java Answers That You Haven’t Seen

This post is by Henn Idan on the Takipi blog.

StackOverflow is a gold mine for developers. It helps us find the most useful answers to specific issues we encounter, and we always find ourselves learning new things from it.

For the following post, we’ve looked into the most popular Java questions and answers and decided to highlight some precious gems that we found. There’s always something new to learn, even if you’re an experienced developer.

Java Answers for All

Java is the second most popular tag on StackOverflow, with more than a million questions linked to it. During the last week, over 4,600 questions were uploaded to the site, and there’s no doubt it’s the biggest and most active online community of developers.

This information goes hand in hand with StackOverflow’s Developer Survey Results for 2016, in which 56,033 coders were asked what’s their language of choice. Java came in at 3rd place:

Most popular technology 2016. Source: Stackoverflow

We already know that Java reigns the job market, and it’s safe to assume that you too have visited StackOverflow once or twice to find an answer for a question. However, even if you’re just casually browsing StackOverflow without a specific question in mind, lots of interesting things can pop up. Let’s see some of those gems.

Branch Prediction

One of the most upvoted Java questions on Stackoverflow is “Why is it faster to process a sorted array than an unsorted array?” In order to answer this, you’ll need to use branch prediction. It’s an architecture that aims to improve the application flow, through guessing the way a specific branch would go before the actual path has been chosen. An educated guess if you’d prefer, only it’s not actually a guess.

The branch here is the ‘if statement.’ In this case, if the array is sorted, branch prediction will work. If it’s not sorted, it won’t work.

Mysticial has tried to explain this in a simpler way, using a railroad and a train. Imagine you operate a junction and need to decided which way the train will go. Will you choose left or right? Sure, you can stop the train and ask the driver which way is the right one, but that makes the whole process slow, clumsy, and irritating. You need to make a guess. How can you be sure your guess is the right one? Take a look at past drives of the current train and understand which way it goes each time.

That’s branch prediction: Identify patterns and follow them.

Unfortunately, in this case the user who asked the main question was a victim of failed branch prediction. That happened because the branch had no recognizable pattern, so trying to predict its actions is pretty random.

Security in Java

Another popular question Java users often upvote is “Why is char[] preferred over String for passwords in Java?” The question itself is a little more specific, asking why a Swing password field has a getPassword() (returns char[]) method instead of getText() (returns String)

No surprise here – it’s a security issue. Strings are immutable, meaning you can’t modify them after they’re created. This also means that you can’t get rid of the data before GC comes knocking. In the off chance someone will have access to your memory, the String with the password might be available for him to take.

That’s why you should use a char array. You’ll be able to explicitly wipe the data when you’re done with it, or you can overwrite it with anything else you’d like. The sensitive data won’t be present anywhere in the system, even before GC runs.

Exceptions

Even though many devs prefer to ignore checked exceptions, there are a lot of questions about exceptions in Java. It’s a main issue you should be addressing in your code, and ignoring the problem won’t make it go away.

One of the most upvoted questions is, “What is a NullPointerException, and how do I fix it?” We weren’t shocked to see how popular this exception is, since it also ranked as the number 1 exception type in production Java applications.

At Takipi, we actually have an option to set up alerts whenever a new NullPointerException (or any other exception) is introduced on the system. Check it out.

Quirks and Magic

Every now and then you come across a puzzling question in StackOverflow that teaches you something new. We chose a few of our favorite gems:

Why Does This Code Using Random Strings Print “hello world”?

The question presents the following print statement, that prints out “hello world”:

The answer is that there is no spoon. Meaning that choosing a random set of integers will not be random. Instead, the instance will follow the random number generation algorithm that begins with a specific seed parameter (in this case -229985452 or -147909649). Every time you’ll ask for a random pattern, that same seed will generate the same pattern – which will print out ‘hello world.’

The user Eng.Fouad explained it perfectly:

In new Random(-229985452).nextInt(27) the first six numbers that the random generates are
8, 5, 12, 12, 15, 0

And the first 6 numbers that new Random(-147909649).nextInt(27) generates are
23, 15, 18, 12, 4, 0

When you add those numbers to the integer representation of the character ` (which is 96), you get “hello world”:

104 –> h
101 –> e
108 –> l
108 –> l
111 –> o

119 –> w
111 –> o
114 –> r
108 –> l
100 –> d

Why Is Subtracting These Two Times (in 1927) Giving a Strange Result?

In the following question, the user parses two date strings referencing times one second apart and compares them.

Instead of getting the result of one, since they are one second apart, he gets the result 353 (queue the spooky music). This has a pretty basic explanation: It’s a time zone thing. On December 31st 1927, Shanghai time moved five minutes and 52 seconds back, and Java is parsing it as the instant for that local date/time.

We do have to point out that if you’ll try to run the code from the original question, it will generate a different result. As Jon Skeet pointed out in his answer, in Time Zone Database Project 2014, the time of the change has moved to 1900-12-31, and it’s now a mere 343 second change.

Uncatchable ChuckNorrisException

This is a bit of an obvious question: If an exception is thrown but no one can catch it, will the application crash? Or as the question asks: “Is it possible to construct a snippet of code in Java that would make a hypothetical java.lang.ChuckNorrisExceptionuncatchable?”

The short answer is that it’s possible, but there’s a “but” involved. You can compile code that throws a ChuckNorrisException, and define a class ChuckNorrisException which does not extend Throwable at runtime. That alone isn’t enough to get it to work, and you’ll have to disable the bytecode verifier. The answer from jtahlborn gives will take you through the full process.

If you’re a fan of Java puzzles, you might want to check out our Java Deathmatchgame.

Hash Maps

One of the most common issue we came across on StackOverflow is related to hash maps. A lot of users want to know what’s the difference between collections and when one should be used over another.

The key ingredient here is iteration order. With HashMap, you’ll have no information about the order, and that order might change as you add more elements to your collection. With TreeMap, you’ll get a sorted iteration, while with LinkedHashMap you’ll get a FIFO order.

If you’re still feeling confused about this, our friends at Rebel Labs made a handy chartthat explains the benefits of one collection over the other.

Final Thoughts

It doesn’t matter how much you know about Java, there’s always more you can learn. StackOverflow helps out with specific problems in the code, but it’s also a great source to learn new information about things we think we know front to back.

If you came across an interesting question, a fiery debate or another quirk, we would love to hear about it in the comments below.

Posted in Information Technology, Software Engineering

Python simplehttpserver Examples

Simple HTTP server in Python3

To server all HTML files from the current working directory

python3 -m http.server

python3 -m http.server 8080            # for different port
python3 -m http.server 8080 --bind 127.0.0.1    # for secure binding

Simple CGI server in Python3

python3 -m http.server --cgi

Simple HTTP server in Python2

To serve all HTML files from the current working directory

python -m simplehttpserver

Simple CGI server in Python2

python -m CGIHTTPServer

 

Posted in Information Technology, Software Engineering

Python subprocess Examples

Syntax

Here is the syntax of important subprocess functions

<status> = subprocess.call         (<command array>[, <options>])
<status> = subprocess.check_call   (<command array>[, <options>])
<string> = subprocess.check_output (<command array>[, <options>])
<Popen>  = subprocess.Popen        (<command array>[, <options>])

Basic Example

import subprocess

subprocess.call(['find', '/etc', '-name', '*.conf'])

In the example a program named ‘find’ is executed without a shell. To launch the command in a shell pass ‘shell=True’ in the list of options

subprocess.call(['find', '/'], shell=True)

Catching STDOUT

To collect all output of a command executed use check_output()

output = subprocess.check_output(["ls"])

Redirecting STDERR

When you only want to redirect STDERR to STDIN and catch this output too, simply add ‘stderr=subprocess.STDOUT’ as an option when calling check_output()

output = subprocess.check_output(["unknown_command"], stderr=subprocess.STDOUT)

Complex Pipe Use Cases

To also handle STDIN, STDOUT and STDERR you need use Popen() and Popen.communicate() to write and read from/to those pipes.

# Launch command with all pipes connected
p = subprocess.Popen(['rm', '-i', '*'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# Pass exactly one 'y' and read output
(out, err) = p.communicate('y')

# Check return code
if p.returncode != 0:
    print("Error!")

Don’t forget to pass a ‘stdxx=subprocess.PIPE’ option for each pipe you want to use

Passing Environment Variables

Popen(["ls"], env={"PATH": "/usr/local/bin"})
Posted in Information Technology, javascript, Software Engineering

Javascript singleton

(1) UPDATE 2019: ES7 Version

class Singleton {
    static instance;

    constructor() {
        if (instance) {
            return instance;
        }

        this.instance = this;
    }

    foo() {
        // ...
    }
}

console.log(new Singleton() === new Singleton());

(2) ES6 Version

class Singleton {
    constructor() {
        const instance = this.constructor.instance;
        if (instance) {
            return instance;
        }

        this.constructor.instance = this;
    }

    foo() {
        // ...
    }
}

console.log(new Singleton() === new Singleton());

Best solution found: http://code.google.com/p/jslibs/wiki/JavascriptTips#Singleton_pattern

function MySingletonClass () {

  if (arguments.callee._singletonInstance) {
    return arguments.callee._singletonInstance;
  }

  arguments.callee._singletonInstance = this;

  this.Foo = function () {
    // ...
  };
}

var a = new MySingletonClass();
var b = MySingletonClass();
console.log( a === b ); // prints: true

For those who want the strict version:

(function (global) {
  "use strict";
  var MySingletonClass = function () {

    if (MySingletonClass.prototype._singletonInstance) {
      return MySingletonClass.prototype._singletonInstance;
    }

    MySingletonClass.prototype._singletonInstance = this;

    this.Foo = function() {
      // ...
    };
  };

var a = new MySingletonClass();
var b = MySingletonClass();
global.result = a === b;

} (window));

console.log(result);
Posted in Devops, Information Technology

ECS provisioning using CloudFormation

You’ll learn how to provision, configure, and orchestrate the EC2 Container Service (ECS) applications into a deployment pipeline that’s capable of deploying new infrastructure and code changes when developers commit changes to a version-control repository so that team members can release new changes to users whenever they choose to do so: Continuous Delivery.
While the primary AWS service described in this solution is ECS, I’ll also be covering the various components and services that support this solution including AWS CloudFormationEC2 Container Registry (ECR), Docker, Identity and Access Management (IAM), VPC and Auto Scaling Services – to name a few. In part 2, I’ll be covering the integration of CodePipeline, Jenkins and CodeCommit in greater detail.
ECS allows you to run Docker containers on Amazon. The benefits of ECS and Docker include the following:

  • Portability – You can build on one Linux operating system and have it work on others without modification. It’s also portable across environment types so you can build it in development and use the same image in production.
  • Scalability – You can run multiple images on the same EC2 instance to scale thousands of tasks across a cluster.
  • Speed – Increase your speed of development and speed of runtime execution.

“ECS is a highly scalable, high performance container management service that supports Docker containers and allows you to easily run applications on a managed cluster of Amazon EC2 instances. Amazon ECS eliminates the need for you to install, operate, and scale your own cluster management infrastructure.” [1] The reason you might use Docker-based containers over traditional virtual machine-based application deployments is that it allows a faster, more flexible, and still very robust immutable deployment pattern in comparison with services such as traditional Elastic Beanstalk, OpsWorks, or native EC2 instances.
While you can very effectively integrate Docker into Elastic Beanstalk, ECS provides greater overall flexibility.
The reason you might use ECS or Elastic Beanstalk containers with EC2 Container Registry over similar offerings such as Docker Hub or Docker Trusted Registry is higher performance, better availability, and lower pricing. In addition, ECR utilizes other AWS services such as IAM and S3, allowing you to compose more secure or robust patterns to meet your needs.
Based on the current implementation of Lambda, the reasons you might choose to utilize ECS instead of serverless architectures include:

  • Lower latency in request response time
  • Flexibility in the underlying language stack to use
  • Elimination of AWS Lambda service limits (requests per second, code size, total code runtime)
  • Greater control of the application runtime environment
  • The ability to link modules in ways not possible with Lambda functions

I’ll be using a sample PHP application provided by AWS to demonstrate Continuous Delivery pipeline using ECS, CloudFormation and, in part 2, AWS CodePipeline.

Create and Connect to a CodeCommit Repository

While you can store your application code in any version-control repository, in this example, I’ll be using the AWS CodeCommit Git repository. I’ll be integrating CodeCommit with CodePipeline. I’m basing the code from the Amazon ECS PHP Simple Demo App located at https://github.com/awslabs/ecs-demo-php-simple-app.
To create your own CodeCommit repo,  follow these instructions: Create and Connect to an AWS CodeCommit Repository. Take note of the repository name as you’ll be using it as a CloudFormation user parameter in part 2. I called my CodeCommit repository ecs-demo. You can call it the same but if you do name it something different, be sure to replace the samples with your repo name.
After you create your CodeCommit repo, copy the contents from the AWS PHP ECS Demo app and commit all of the files.
CodeCommit provides the following features and benefits[2]:

  • Highly available, Secure and Private Git repositories
  • Use your existing Git tools
  • Automatically encrypts all files in transit and at rest
  • Provides Webhooks – to trigger Lambda functions or push notifications in response to events
  • Integrated with other AWS services like IAM so you can define user-specific permissions

Create a Private Image Repository in ECS using ECR

codepipeline_ecr_archYou can create private Docker repositories using ECS Repositories (ECR) to store your Docker images. Follow these instructions to manually create an ECR: Create a Repository.
A snippet of the CloudFormation template for provisioning an ECR repo is listed below.

    "MyRepository":{
      "Type":"AWS::ECR::Repository",
      "Properties":{
        "RepositoryName":{
          "Ref":"AWS::StackName"
        },
        "RepositoryPolicyText":{
          "Version":"2008-10-17",
          "Statement":[
            {
              "Sid":"AllowPushPull",
              "Effect":"Allow",
              "Principal":{
                "AWS":[
                  {
                    "Fn::Join":[
                      "",
                      [
                        "arn:aws:iam::",
                        {
                          "Ref":"AWS::AccountId"
                        },
                        ":user/",
                        {
                          "Ref":"IAMUsername"
                        }
                      ]
                    ]
                  }
                ]
              },
              "Action":[
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability",
                "ecr:PutImage",
                "ecr:InitiateLayerUpload",
                "ecr:UploadLayerPart",
                "ecr:CompleteLayerUpload"
              ]
            }
          ]
        }
      }
    }

In defining an ECR, you can securely store your Docker images and refer to them when building, tagging and pushing these Docker images.
To launch the CloudFormation stack to create an ECR repository, click this button: . Your IAM username is a parameter to this CloudFormation template. You only need to enter the IAM username (and not the entire ARN) as the input value. Make note of the ECSRepository Output from the stack as you’ll be using this as an input to the ECS Environment Stack in part 2.

Docker

“Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.” [3] In this demonstration, you’ll build, tag and push a PHP application as a Docker image into an ECR repository.

Build Docker Image and Upload to ECR Locally

Prerequisites

  • You’re running these commands from an Amazon Linux EC2 instance. If you’re not, you’ll need to adapt the instructions according to your OS flavor.
  • You’ve created an ECR repo (see the “Create a Private Image Repository in ECS using ECR” section above)
  • You’ve created a CodeCommit repository and committed the PHP code from the AWS PHP app in GitHub (see the “Create and Connect to a CodeCommit Repository” section above)

Steps

  1. Install Docker on an Amazon Linux EC2 instance for which your AWS CLI has been configured (you can find detailed instructions at Install Docker)
    sudo yum update -y
    sudo yum install -y docker
    sudo service docker start
    sudo usermod -a -G docker ec2-user
  2. Logout and log back in and type:
    docker info
  3. Install Git:
    sudo yum -y install git*
  4. Clone the ECS PHP example application (if you used a different repo name, be sure to update the sample command here):
    git clone ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/ecs-demo
  5. Change your directory:
    cd ecs-demo
  6. Configure your AWS account by running the command below and following the prompts to enter your credentials, region and output format.
    aws configure
  7. Run the command below to login to ECR.
    eval $(aws --region us-east-1 ecr get-login)
  8. Build the image using Docker. Replace REPOSITORY_NAME with the ECSRepository Output from the ECR stack you launched and TAG with a unique value. Make note of the name the image tag you’re using in creating the Docker image as you’ll be using it as a input parameter to a CloudFormation stack later. If you want to use the default value, just name it latest.
    docker build -t REPOSITORY_NAME:TAG .
  9. Tag the image (replace REPOSITORY_NAME, TAG and AWS_ACCOUNT_ID):
    docker tag REPOSITORY_NAME:TAG AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/REPOSITORY_NAME:TAG
  10. Push the tagged image to ECR (replace REPOSITORY_NAMEAWS_ACCOUNT_ID and TAG):
    docker push AWS_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/REPOSITORY_NAME:TAG
  11. Verify the image was uploaded to your ECS Repository by going to your AWS ECS Console, clicking on Repositories and selecting the repository you created when you launched the ECS Stack.

Dockerfile

“A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build users can create an automated build that executes several command-line instructions in succession.” [4] The snippet you see below is the Dockerfile to run the PHP sample application. You can see that it runs OS updates, installs the required packages including apache and PHP and then configures the HTTP server and port. While these are types of steps you might run in any automated build and deployment script, the difference is that it’s running these steps within a container which means that it runs very quickly, you can run these same steps across operating systems, and you can run these procedures across multiple tasks in a cluster.

FROM ubuntu:12.04
# Install dependencies
RUN apt-get update -y
RUN apt-get install -y git curl apache2 php5 libapache2-mod-php5 php5-mcrypt php5-mysql
# Install app
RUN rm -rf /var/www/*
ADD src /var/www
# Configure apache
RUN a2enmod rewrite
RUN chown -R www-data:www-data /var/www
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
EXPOSE 80
CMD ["/usr/sbin/apache2", "-D",  "FOREGROUND"]

This Dockerfile gets run when you run the docker build command. This file has been committed to my CodeCommit repo as you can see in the figure below.

ecs_codecommit

AWS CodeCommit repository for a PHP application illustrating Dockerfile location

Create an ECS Environment in CloudFormation

In this section, I’m describing the how to configure the entire ECS stack in CloudFormation. This includes the architecture, its dependencies, and the key CloudFormation resources that make up the stack.

Architecture

The overall solution architecture is illustrated in the CloudFormation diagram below.

codepipeline_ecs_arch.jpg

Provisioning, Configuring and Orchestrating an EC2 Container Service Architecture

  • Auto Scaling Group – I’m using an auto scaling group to scale the underlying EC2 infrastructure in the ECS Cluster. It’s used in conjunction with the Launch Configuration.
  • Auto Scaling Launch Configuration – I’m using a launch configuration to scale the underlying EC2 infrastructure in the ECS Cluster. It’s used in conjunction with the  Auto Scaling Group.
  • CodeCommit – I’m using CodeCommit as my Git repo to store the application and infrastructure code.
  • CodePipeline – CodePipeline describes my Continuous Delivery workflow. In particular, it integrates with CodeCommit and Jenkins to run actions every time someone commits new code to the CodeCommit repo. This will be covered in more detail in part 2.
  • ECS Cluster – “An ECS cluster is a logical grouping of container instances that you can place tasks on.”[6]
  • ECS Service – With an ECS service, you can run a specific number of instances of a task definition simultaneously in an ECS cluster [5]
  • ECS Task Definition – A task definition is the core resource within ECS. This is where you define which Docker images to run, CPU/Memory, ports, commands and so on. Everything else in ECS is based upon the task definition
  • Elastic Load Balancer – The ELB provides the endpoint for the application. The ELB dynamically determines which EC2 instance in the cluster is serving the running ECS tasks at any given time.
  • IAM Instance Profile – “An instance profile is a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.” [7] In the sample, I’m using the instance profile to define the roles for which launch configurations use as part of the underlying EC2 instance that the ECS cluster runs
  • IAM Roles – I’m describing roles that have access to certain AWS resources for the EC2 instances (for ECS), Jenkins and CodePipeline
  • Jenkins – I’m using Jenkins to execute the actions that I’ve defined in CodePipeline. For example, I have a bash script that updates the CloudFormation stack when an ECS Service is update. This action is orchestrated via CodePipeline and then executed on te Jenkins server on one of its configured jobs. This will be covered in more detail in part 2.
  • Virtual Private Cloud (VPC) – In the CloudFormation template, I’m using a VPC template that we developed to define VPC resources such as: VPCGatewayAttachment, SecurityGroup, SecurityGroupIngress, SecurityGroupEgress, SubnetNetworkAclAssociation, NetworkAclEntry, NetworkAcl, SubnetRouteTableAssociation, Route, RouteTable, InternetGateway, and Subnet

Dependencies

There are four core dependencies in this solution: EC2 Key Pair, CodeCommit Repo, a VPC, and an ECR repo and Docker Image

  • EC2 Key Pair – A key pair for which you have access. See Create a Key Pair.
  • CodeCommit – In this demo, I’m using an AWS CodeCommit Git repo to store the PHP application code along with my Docker configuration. See the instructions for configuring a Git repo in CodeCommit above
  • VPC – This template requires an existing AWS Virtual Private Cloud has been created
  • ECR repo and image – You should have created an E2 Container Service Repository (ECR) using the CloudFormation template from the previous section. You should have also built, tagged and pushed a Docker image to ECR using the instructions described at Create a Private Image Repository in ECS using ECR above

ECS Cluster

With an ECS Cluster, you can manage multiple services. An ECS Container Instance runs an ECS agent that is registered to the ECS Cluster. To define an ECS Cluster in CloudFormation, use the Cluster resource: AWS::ECS::Cluster as shown below.

    "EcsCluster":{
      "Type":"AWS::ECS::Cluster",
      "DependsOn":[
        "MyVPC"
      ]
    },

ECS Service

An ECS Service defines a task definition and a desired number of task instances. A service manages tasks of a specified task definition.
In the context of ECS, an ELB distributes load between the different EC2 instances hosting your tasks, so you can optionally create a new ELB when creating a service.
To define an ECS Service in CloudFormation, use the Service resource: AWS::ECS::Service.

    "EcsService":{
      "Type":"AWS::ECS::Service",
      "DependsOn":[
        "MyVPC",
        "ECSAutoScalingGroup"
      ],
      "Properties":{
        "Cluster":{
          "Ref":"EcsCluster"
        },
        "DesiredCount":"1",
        "DeploymentConfiguration":{
          "MaximumPercent":100,
          "MinimumHealthyPercent":0
        },
        "LoadBalancers":[
          {
            "ContainerName":"php-simple-app",
            "ContainerPort":"80",
            "LoadBalancerName":{
              "Ref":"EcsElb"
            }
          }
        ],
        "Role":{
          "Ref":"EcsServiceRole"
        },
        "TaskDefinition":{
          "Ref":"PhpTaskDefinition"
        }
      }
    },

Notice that I defined a DeploymentConfiguration with a MinimumHealthyPercent of 0. Since I’m only using one EC2 instance in development, the ECS service would fail during a CloudFormation update so by setting the MinimumHealthyPercent to zero, the application will experience a few seconds of downtime during the update. Like most applications/services these days, if I need to continual uptime, I’d increase the number of instances in my Auto Scaling Group and increase the MinimumHealthyPercent property.

Task Definition

With an ECS Task Definition, you can define multiple Container Definitions and volumes. With a Container Definition, you define port mappings, environment variables, CPU Units and Memory. An ECS Volume is a persistent volume to mount and map to container volumes.
To define an ECS Task Definition, use the ECS Task Definition resource: AWS::ECS::TaskDefinition.

    "PhpTaskDefinition":{
      "Type":"AWS::ECS::TaskDefinition",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "ContainerDefinitions":[
          {
            "Name":"php-simple-app",
            "Cpu":"10",
            "Essential":"true",
            "Image":{
              "Fn::Join":[
                "",
                [
                  {
                    "Ref":"AWS::AccountId"
                  },
                  ".dkr.ecr.us-east-1.amazonaws.com/",
                  {
                    "Ref":"ECSRepoName"
                  },
                  ":",
                  {
                    "Ref":"ImageTag"
                  }
                ]
              ]
            },
            "Memory":"300",
            "PortMappings":[
              {
                "HostPort":80,
                "ContainerPort":80
              }
            ]
          }
        ],
        "Volumes":[
          {
            "Name":"my-vol"
          }
        ]
      }
    },

Auto Scaling

To define an Auto Scaling Group, use the Auto Scaling Group resource in CloudFormation: AWS::AutoScaling::AutoScalingGroup.

    "ECSAutoScalingGroup":{
      "Type":"AWS::AutoScaling::AutoScalingGroup",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "VPCZoneIdentifier":[
          {
            "Ref":"publicSubnet01"
          },
          {
            "Ref":"publicSubnet02"
          }
        ],
        "LaunchConfigurationName":{
          "Ref":"ContainerInstances"
        },
        "MinSize":"1",
        "MaxSize":{
          "Ref":"MaxSize"
        },
        "DesiredCapacity":{
          "Ref":"DesiredCapacity"
        }
      },
      "CreationPolicy":{
        "ResourceSignal":{
          "Timeout":"PT15M"
        }
      },
      "UpdatePolicy":{
        "AutoScalingRollingUpdate":{
          "MinInstancesInService":"1",
          "MaxBatchSize":"1",
          "PauseTime":"PT15M",
          "WaitOnResourceSignals":"true"
        }
      }
    },

To define a Launch Configuration, use the Launch Configuration resource in CloudFormation: AWS::AutoScaling::LaunchConfiguration.

    "ContainerInstances":{
      "Type":"AWS::AutoScaling::LaunchConfiguration",
      "DependsOn":[
        "MyVPC"
      ],
      "Metadata":{
        "AWS::CloudFormation::Init":{
          "config":{
            "commands":{
              "01_add_instance_to_cluster":{
                "command":{
                  "Fn::Join":[
                    "",
                    [
                      "#!/bin/bash\n",
                      "echo ECS_CLUSTER=",
                      {
                        "Ref":"EcsCluster"
                      },
                      " >> /etc/ecs/ecs.config"
                    ]
                  ]
                }
              }
            },
            "files":{
              "/etc/cfn/cfn-hup.conf":{
                "content":{
                  "Fn::Join":[
                    "",
                    [
                      "[main]\n",
                      "stack=",
                      {
                        "Ref":"AWS::StackId"
                      },
                      "\n",
                      "region=",
                      {
                        "Ref":"AWS::Region"
                      },
                      "\n"
                    ]
                  ]
                },
                "mode":"000400",
                "owner":"root",
                "group":"root"
              },
              "/etc/cfn/hooks.d/cfn-auto-reloader.conf":{
                "content":{
                  "Fn::Join":[
                    "",
                    [
                      "[cfn-auto-reloader-hook]\n",
                      "triggers=post.update\n",
                      "path=Resources.ContainerInstances.Metadata.AWS::CloudFormation::Init\n",
                      "action=/opt/aws/bin/cfn-init -v ",
                      "         --stack ",
                      {
                        "Ref":"AWS::StackName"
                      },
                      "         --resource ContainerInstances ",
                      "         --region ",
                      {
                        "Ref":"AWS::Region"
                      },
                      "\n",
                      "runas=root\n"
                    ]
                  ]
                }
              }
            },

>

IAM

To define an IAM Instance Profile, use the InstanceProfile resource in CloudFormation: AWS::IAM::InstanceProfile.

    "EC2InstanceProfile":{
      "Type":"AWS::IAM::InstanceProfile",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "Path":"/",
        "Roles":[
          {
            "Ref":"EC2Role"
          }
        ]
      }
    },
    "JenkinsRole":{
      "Type":"AWS::IAM::Role",
      "Properties":{
        "AssumeRolePolicyDocument":{
          "Statement":[
            {
              "Sid":"",
              "Effect":"Allow",
              "Principal":{
                "Service":"ec2.amazonaws.com"
              },
              "Action":"sts:AssumeRole"
            }
          ]
        },
        "Path":"/"
      }
    },

To define an IAM Role, use the IAM Role resource in CloudFormation: AWS::IAM::Role. The snippet below is for the EC2 role.

    "EC2Role":{
      "Type":"AWS::IAM::Role",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "AssumeRolePolicyDocument":{
          "Statement":[
            {
              "Effect":"Allow",
              "Principal":{
                "Service":[
                  "ec2.amazonaws.com"
                ]
              },
              "Action":[
                "sts:AssumeRole"
              ]
            }
          ]
        },
        "Path":"/",
        "Policies":[
          {
            "PolicyName":"ecs-service",
            "PolicyDocument":{
              "Statement":[
                {
                  "Effect":"Allow",
                  "Action":[
                    "ecs:CreateCluster",
                    "ecs:RegisterContainerInstance",
                    "ecs:DeregisterContainerInstance",
                    "ecs:DiscoverPollEndpoint",
                    "ecs:Submit*",
                    "ecr:*",
                    "ecs:Poll"
                  ],
                  "Resource":"*"
                }
              ]
            }
          }
        ]
      }
    },

The snippet below is for defining the ECS IAM role.

    "EcsServiceRole":{
      "Type":"AWS::IAM::Role",
      "Properties":{
        "AssumeRolePolicyDocument":{
          "Statement":[
            {
              "Effect":"Allow",
              "Principal":{
                "Service":[
                  "ecs.amazonaws.com"
                ]
              },
              "Action":[
                "sts:AssumeRole"
              ]
            }
          ]
        },
        "Path":"/",
        "Policies":[
          {
            "PolicyName":"ecs-service",
            "PolicyDocument":{
              "Statement":[
                {
                  "Effect":"Allow",
                  "Action":[
                    "elasticloadbalancing:Describe*",
                    "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                    "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                    "ec2:Describe*",
                    "ec2:AuthorizeSecurityGroupIngress"
                  ],
                  "Resource":"*"
                }
              ]
            }
          }
        ]
      }
    },

EC2

To define security group ingress within a VPC, use the SecurityGroupIngress resource in CloudFormation: AWS::EC2::SecurityGroupIngress.

    "InboundRule":{
      "Type":"AWS::EC2::SecurityGroupIngress",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "IpProtocol":"tcp",
        "FromPort":"0",
        "ToPort":"65535",
        "SourceSecurityGroupId":{
          "Fn::GetAtt":[
            "TargetSG",
            "GroupId"
          ]
        },
        "GroupId":{
          "Fn::GetAtt":[
            "TargetSG",
            "GroupId"
          ]
        }
      }
    },

To define the security group egress within a VPC, use theSecurityGroupEgress resource in CloudFormation: AWS::EC2::SecurityGroupEgress.

    "OutboundRule":{
      "Type":"AWS::EC2::SecurityGroupEgress",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "IpProtocol":"tcp",
        "FromPort":"0",
        "ToPort":"65535",
        "DestinationSecurityGroupId":{
          "Fn::GetAtt":[
            "TargetSG",
            "GroupId"
          ]
        },
        "GroupId":{
          "Fn::GetAtt":[
            "SourceSG",
            "GroupId"
          ]
        }
      }
    },

To define the security group within a VPC, use the SecurityGroup resource in CloudFormation: AWS::EC2::SecurityGroup.

    "SourceSG":{
      "Type":"AWS::EC2::SecurityGroup",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "VpcId":{
          "Ref":"MyVPC"
        },
        "GroupDescription":"Sample source security group",
        "SecurityGroupIngress":[
          {
            "IpProtocol":"tcp",
            "FromPort":"80",
            "ToPort":"80",
            "CidrIp":"0.0.0.0/0"
          }
        ],
        "Tags":[
          {
            "Key":"Name",
            "Value":{
              "Fn::Join":[
                "",
                [
                  {
                    "Ref":"AWS::StackName"
                  },
                  "-SourceSG"
                ]
              ]
            }
          }
        ]
      }
    },

ELB

To define the ELB, use the LoadBalancer resource in CloudFormation: AWS::ElasticLoadBalancing::LoadBalancer.

    "EcsElb":{
      "Type":"AWS::ElasticLoadBalancing::LoadBalancer",
      "DependsOn":[
        "MyVPC"
      ],
      "Properties":{
        "Subnets":[
          {
            "Ref":"publicSubnet01"
          },
          {
            "Ref":"publicSubnet02"
          }
        ],
        "Listeners":[
          {
            "LoadBalancerPort":"80",
            "InstancePort":"80",
            "Protocol":"HTTP"
          }
        ],
        "SecurityGroups":[
          {
            "Ref":"SourceSG"
          },
          {
            "Ref":"TargetSG"
          }
        ],
        "HealthCheck":{
          "Target":"HTTP:80/",
          "HealthyThreshold":"2",
          "UnhealthyThreshold":"10",
          "Interval":"30",
          "Timeout":"5"
        }
      }
    },

Summary

In this first part of the series, you learned how to use CloudFormation to fully automate the provisioning of the EC2 Container Service and Docker which includes ELB, Auto Scaling, and VPC resources. You also learned how to setup a CodeCommit repository.
In the next and last part of this series, you’ll learn how to orchestrate all of the changes into a deployment pipeline to achieve Continuous Delivery using CodePipeline and Jenkins so that any change made to the CodeCommit repo can be deployed to production in an automated fashion. I’ll provide access to all the code resources in part 2 of this series. Let us know if you have any comments or questions @stelligent or @paulduvall.
Stelligent is hiring! Do you enjoy working on complex problems like figuring out ways to automate all the things as part of a deployment pipeline? Do you believe in the “everything-as-code” mantra? If your skills and interests lie at the intersection of DevOps automation and the AWS cloud, check out the careers page on our website.

Resources

Here’s a list of some of the resources described in this post:

source:

Automating ECS: Provisioning in CloudFormation (Part 1)

 

Posted in Information Technology, Software Engineering

Ruby Regex Examples

Some simple examples for using regular expressions in Ruby scripts.

Regular Expressions

1. Syntax

Compared to other scripting languages Ruby more behaves like Perl allowing to use regex seemlessly without the need for explicit objects in Python or a function in PHP. So you can just match something with putting a regular expression between two slashes:

/search string/

2. Simple String Checks

Here are some syntax examples that check strings for certain content:

Basic Matching

text =~ /Ruby/       # Match for an unbound literal
text =~ /^Ruby/      # Match literal at start of string
text =~ /Ruby$/      # Match literal at end of string
text =~ /^Ruby$/     # Match for exact string content
text =~ /^$/         # Match empty string

Using different regex delimiters

%r/Ruby/               # / as commonly used delimiter
%r@Ruby@               # @ as delimiter
%r!Ruby!               # ! as delimiter

Changing the delimiter becomes useful in some cases

"http://" =~ /http:\/\//     # match http:// protocol prefix with / delimiter
"http://" =~ %r#http://#     # match http:// protocol prefix with # delimiter

Case sensitity

text =~ /Ruby/                # case sensitive string matching
text =~ /ruby/i               # case in-sensitive string matching

Matching with wildcards

"Ruby" =~ /R..y/            # match a single character with .
"Ruby" =~ /R.*y/            # match multipe characters
"Ruby" =~ /R[a-z]y/         # match from character range a-z
"Rbuy" =~ /[Ruby]*/         # match from character set R,u,b and y
"Ruby" =~ /R\wy/            # match one word character
"regex in Ruby" =~ /\bRuby\b/"   # match the word "Ruby", but not "Ruby" as larger string

Using quantifiers

"Ruby" =~ /[Ruby]{4}/          # match exactly 4 characters from set [Ruby]
"Ruby" =~ /[Ruby]{4,4}/        # match exactly 4 characters from set [Ruby]
"Ruby" =~ /[Ruby]{,3}/         # match at most 3 characters from set [Ruby]
"Ruby" =~ /[Ruby]{3,}/         # match at least 3 characters from set [Ruby]

3. Replacing Patterns

You need to do substitution using the in place string object methods sub!() for replacing first occurence and gsub!() for replacing all occurences:

text.sub!(/Rbuy/, "Ruby")

4. Capture Groups

To extract data using regular expression we have to use capture/grouping syntax and

  • to do exactly one match: the String#match method and MatchData#captures to produce a result array
  • to do multiple matches: the String#scan method which returns a nested array with an result array for each match

Some basic examples to do the exactly one match with String#match

# Extract everything after the literal "START"
if result = line.match(/START(.*)/")
   text = result.captures
end
# Extract the number from a date string "2012-10-20"
if result = line.match(/(\d{4})-(\d{2})-(\d{2})/)
  year, week, day = result.captures
end
# Nesting of capture groups, extract full name, and both parts... string "name is Doe, John"
if result = line.match(/name is ((\w+), (\w+))/)
  fullname, firstname, lastname = result.capture
end

Ensure always to check for nil result if the match might fail!

Posted in Information Technology

Regex Overview Examples

Usage in different languages

As the actual use of regular expressions is quite language specific there are separate cheat sheets:

Standards

Regex Variants

Variant Used By Meta Characters Back References
POSIX BRE grep, sed ^ $ \{\} \[\] \(\) . * GNU extension: \+ \? \1 … \9
POSIX ERE egrep, awk, gawk ^ $ {} [] () . * + ? | gawk: \\1 .. \\9
Perl RE Perl, C, PHP, Python, Ruby… Every programming language using PCRE ^ $ {} () . * + ? | Based on language

Perl \g1 \g2 … \g-1 \g-2 … \g{-1} \g{-2} … \g{name} \k{name} \g’name’ \k’name’
PHP \\1 .. \\9 $1 .. $99 ${1} .. ${99}
Python \1 .. \99 \g1 .. \g99 \g (?P…) (?P=name)

Other Hints:

Posted in Devops, Information Technology

Perl Regex Examples

Syntax

Perl Regex Reference

Misc

https://lzone.de/examples/Perl%20Regex

 

Posted in Information Technology, Software Engineering

CSS Cheat Sheet

Ressources

  • CSS Layouts Introduction
  • Web – Javascript Online IDE: JSFiddle
  • External Link Indicators
    // Set for all links
    .content a[href^="//"]:after, 
    .content a[href^="http://"]:after, 
    .content a[href^="https://"]:after {
        content: url(/images/Icon_External_Link.png);
        margin: 0 0 0 5px;
    }
    
    // Exclude local links
    .content a[href^="//lzone.de/"]:after, 
    .content a[href^="http://lzone.de/"]:after, 
    .content a[href^="https://lzone.de/"]:after {
        content: '';
        margin: 0;
    }
    
  • Image Inlining
    <img src="data:image/png;base64,[IMAGE_DATA_STRING]" />
  • CSS Pre-Processors

Advanced CSS Style Overview

CSS 1 No Wrapping
white-space: nowrap;
CSS 1 Inline Block Layout
display: inline-block;
CSS 2 Max Width
max-width: 1024px;
CSS 2 Table – Border Collapse
border-collapse: collapse
CSS 2 Insert Before/After
p.chapter:before { content: 'something'; }

a.href[href^="http://"]:after {
   content: url(/images/Icon_External_Link.png);
}
CSS 3 nth Child
tr:nth-child(odd) { }
tr:nth-child(even) { }
tr:nth-child(5) { }       // just the 5th
tr:nth-child(2n+3) { }    // every 2nd starting at 3
tr:nth-child(-n+4) { }    // first 4
CSS 3 Media Queries
<link rel="stylesheet" media="(max-width: 800px)" href="example.css" />
@media (min-width: 400px) { }
@media (max-width: 600px) { }
@media (max-width: 600px) and (orientation: landscape) { }
@media screen and (min-aspect-ratio: 16/9) { }
@media print and (min-resolution: 300dpi) { }
CSS 3 View Port
<meta name="viewport" content="width=320" />

@-viewport { width: 640px; }

@-ms-viewport { width: device-width }

@media (max-width: 699px) and (min-width: 520px) {
  @-viewport {
    width: 640px;
  }
}

@viewport {
    width: 980px;
    min-zoom: 0.25;
    max-zoom: 5;
    orientation: landscape;
}
CSS 3 Background Gradient
background: -webkit-linear-gradient(#444, #000);
background: -o-linear-gradient(#444, #000);
background: -moz-linear-gradient(#444, #000);
background: linear-gradient(#444, #000);
CSS 3 Border Radis
border-radius: 6px;
border-top-left-radius: 6px;
border-top-right-radius: 6px;
border-bottom-right-radius: 6px;
border-bottom-left-radius: 6px; 
CSS 3 div – Colum Count
-webkit-column-count: 3;
-moz-column-count: 3;
column-count: 3;
CSS 3 Animations
animation-name: spin; 
animation-duration: 5s;
animation-iteration-count: infinite; 
animation-timing-function: linear;
CSS 3 Circle Drawing Circles are done using a 50% border radius

border-radius:50%;
Draft Flexbox
display: flex;
flex-direction: <row | row-reverse | column | column-reverse>;
flex-wrap: <nowrap | wrap | wrap-reverse>;
...
CSS 3 Font Smoothing
body {
    -webkit-font-smoothing: antialiased;
    -moz-osx-font-smoothing: grayscale;
}

 

Posted in Information Technology, Software Engineering

What is GraphQL Schema Language?

What is GraphQL Schema Language?

It is a shorthand notation to succinctly express the basic shape of your GraphQL schema and its type system.

Example of a typical GraphQL schema expressed in shorthand.

interface Entity {
  id: ID!
  name: String
}
type User implements Entity {
  id: ID!
  name: String
  age: Int
  balance: Float
  is_active: Boolean
  friends: [User]!
}
type Root {
   me: User
   users(limit: Int = 10): [User]!
   friends(forUser: ID!, limit: Int = 5): [User]!
}
schema {
  query: Root
  mutation: ...
  subscription: ...
}

(Ain’t it purrtyyy?)


Basics

Schema
=======
GraphQL Schema => schema
Built-in scalar types
=====================
GraphQL Int     => Int
GraphQL Float   => Float
GraphQL String  => String
GraphQL Boolean => Boolean
GraphQL ID      => ID
Type Definitions
================
Scalar Type        => scalar
Object Type        => type
Interface Type     => interface
Union Type         => union
Enum Type          => enum
Input Object Type  => input
Type Markers
============
Non-null Type                    => <type>!     e.g String!
List Type                        => [<type>]    e.g [String]
List of Non-null Types           => [<type>!]   e.g [String!]
Non-null List Type               => [<type>]!   e.g [String]!
Non-null List of Non-null Types  => [<type>!]!  e.g [String!]!

Examples

Below are further examples to illustrate how we can use the above Shorthand Notation to describe our schema.

Input Arguments

Basic Input
============
type Root {
  users(limit: Int): [User]!
}

Input with default value
=========================
type Root {
  users(limit: Int = 10): [User]!
}

Input with multiple args
=========================
type Root {
  users(limit: Int, sort: String): [User]!
}
Input with multiple args and default values
===========================================
type Root {
  users(limit: Int = 10, sort: String): [User]!
}
or
type Root {
  users(limit: Int, sort: String = "asc" ): [User]!
}
or
type Root {
  users(limit: Int = 10, sort: String = "asc" ): [User]!
}

Interfaces

  • Object implementing an Interface
interface Foo {
  is_foo: Boolean
}
type Bar implements Foo {
  is_foo: Boolean
  is_bar: Boolean
}
  • Object implementing multiple Interfaces
interface Foo {
  is_foo: Boolean
}
interface Goo {
  is_goo: Boolean
}
type Bar implements Foo, Goo {
  is_bar: Boolean
  is_foo: Boolean
  is_goo: Boolean
}

Unions

  • Union of a single Object
type Foo {
  name: String
}
union SingleUnion = Foo
type Root {
  single: SingleUnion
}
  • Union of multiple Objects
type Foo {
  name: String
}
type Bar {
  is_bar: String
}
union MultipleUnion = Foo | Bar
type Root {
  multiple: MultipleUnion
}

Enums

enum RGB {
  RED
  GREEN
  BLUE
}
type Root {
  color: RGB
}

Input Object Types

input ListUsersInput {
  limit: Int 
  since_id: ID
}
type Root {
  users(params: ListUsersInput): [Users]!
}

Custom Scalar

scalar Url
type User {
  name: String
  homepage: Url
}

There you go, all you need to know about GraphQL’s shorthand notation. (Probably? Let me know if I had missed anything out)

Now go forth, and wield the power that you have just been bestowed upon with, responsibly.


Extras

  1. Head over to the GitHub repo (https://github.com/sogko/graphql-shorthand-notation-cheat-sheet) for download links (PDF and PNG available)
  2. utilities/schemaPrinter.js
    graphql-js comes with an utility to transform your schema into a shorthand notation representation.
  3. Check out the official documentation on GraphQL Type System.

cheat sheet:

https://wehavefaces.net/graphql-shorthand-notation-cheatsheet-17cd715861b6