codemonth blog codemonth blog - One project every month - making stuff better. Mon, 20 Feb 2017 08:06:08 +0700 Mon, 20 Feb 2017 08:06:08 +0700 30 Updated random_ninja and testdata_ninja packages 48C7A67F9002EA7AE050007F010061CE Sat, 18 Feb 2017 04:47:39 +0700 <p>A couple of people have requested that I explain how to install the entire random_ninja and testdata_ninja packages manually. I have created a gist here with the full and complete order of the files: <a href="">Full install list</a>. So download <a href="">random_ninja zipfile</a> and <a href="">testdata_ninja zipfile</a> and follow the order of the gist to install all the code.</p> <p>Once you have installed the packages you have the full random library available, which you can read in full detail about here: <a href="">Post #1</a>, <a href="">Post #2</a> and <a href="">Post #3</a>. For testdata package you can see a quick demo <a href="">here</a> and <a href="">here</a>.</p> <p>Release 3 of random_ninja is coming up and this is the list of new functionality that will be fully supported (HINT: The code is already in the github source): <ul> <li><strong>company_random</strong> <ul> <li><strong>r_companyname</strong> - generate a random company name.</li> <li><strong>r_industry</strong> - generate a random industry type.</li> <li><strong>r_companyid</strong> - generate a random country tax id.</li> <li><strong>r_employees</strong> - generate a random employee count.</li> <li><strong>r_revenue</strong> - generate a random revenue number.</li> </ul> </li> <li><strong>phone_random</strong> <ul> <li><strong>r_brand</strong> - generate a random brand name.</li> </ul> </li> <li><strong>computer_random</strong> <ul> <li><strong>r_error</strong> - generate a random error string.</li> </ul> </li> <li><strong>util_random</strong> <ul> <li><strong>ru_permute</strong> - Permute a string: Morten -> Metron.</li> <li><strong>ru_scramble</strong> - Scramble a string: Morten -> Diwnah.</li> <li><strong>ru_obfuscate</strong> - Obfuscate a string: Morten -> FasNituha.</li> </ul> </li> <li><strong>transport_random</strong> <ul> <li><strong>r_vehicle_registration</strong> - generate random country specific vehicle registration plate.</li> <li><strong>r_icao</strong> - generate a random country specific aircraft registration code.</li> <li><strong>r_imo</strong> - generate a random vessel registration code.</li> </ul> </li> </ul> </p> <p>Also there are 3 new locale supported for names, addresses and more. These countries are Denmark, China and Dhubai. Performance improvements for Markov text generation are coming as well as new SWIFT and FIX financial random generation.</p> plsql promises - chaining and thenables 464C11564C15BB9DE050007F01003BC2 Sat, 21 Jan 2017 08:39:45 +0700 <p>So last blog entry <a href="">Do or do not.</a> on the promises library, covered the basics of what promises can give us.</p><p>This entry will cover more around chaining of promises and how you can easily run your code in parallel and still make sure they run in the right order, when needed.</p><p>Promises by definition has a method called <b>[then]</b>. What this method does, is that it will take the value of the promise once it is fulfilled, and use that as input to a new function defined by us. The input to then is, either a function to run when fulfilled or a function to run if rejected or both. Although not in the standard there are more and more libraries implementing a method called <b>[done]</b>. It has the same input options as the <b>[then]</b> method but it does not return a new promise. So only use this method when you do not care about the result of the chain call. Since we are in Oracle I cannot name the method <b>[then]</b> since it is a reserved word, so I had to add an _f to the name, so it becomes <b>[then_f]</b>.</p><p>So here is the first example. We create a promise based on the function p_multiplier, that takes a number and multiplies it by 2. We then say that if this promise is fulfilled we want to execute the function called p_multi_convert. This function takes the previous promise result, and will return a text that will tell us if the output is less or more than 50.</p><pre> declare test_promise promise; begin test_promise := promise('p_multiplier', 42); test_promise := test_promise.then_f('p_multi_convert'); promises_ninja.promise_wait(test_promise); dbms_output.put_line('Value of test_promise2 is: ' || test_promise.getanyvalue); end; / </pre> Do, or do not. There is no "try". 456EF39023E4FEDCE050007F01006B83 Fri, 06 Jan 2017 10:44:20 +0700 <p>About 7 months ago, I was reading an article about some of the new features that were in javascript/ecma/coffeescript or whatever it is called now. One of the things that stuck out, was <a href="" location="_new">promises</a>. I liked the idea of "asynchronous" execution with a possible chained dependency, that you could just define and run, and then carry on with other tasks without having to wait for the result. What you do instead, is you receive a "promise" of the execution. A promise that at some point will contain the result from your call. </p><p>The Oracle database has the capability to do this already in dbms_scheduler using chains, but it is not dynamic and the complete flow has to be defined up front.</p><p>So I decided to try and see if this could be done in plsql somehow ... And then I got occupied with something else. Instead I picked up the idea over christmas, and decided to give it a go. What I have today is a 60% implementation, that should get to 100% over the next 2 weeks, but the stuff that is here now, is functional and you can do some fun pieces of code with it</p><p>Let me just give you a couple of samples of what <a href="">PROMISES_NINJA</a> is capable of:</p> Demographically correct test data 41160789E353E6F4E050007F01004D39 Sun, 13 Nov 2016 03:11:20 +0700 <p>One of the main reasons behind finishing versions <a href="">1.5.0 of RANDOM_NINJA</a>, was to be able to add the localization for different countries so that I could create test data sets, that are demographically correct.</p><p>So after adding that, the TESTDATA_NINJA package, can now create statistically correct data sets for 3 countries: <ul> <li>United States</li> <li>China</li> <li>Denmark</li> </ul></p><p>The following ratio of data will be correct according to UN Statistics and CIA World Book: <ul> <li><strong>Age group</strong> - Age ratios will be divided in 0-14, 15-64 and 65+</li> <li><strong>Female/Male</strong> - Gender rations will be divided within the different age groups, according to statistics</li> </ul> </p><p>Identification numbers and birthdays will also be in statistically correct ratios and in valid formats.</p> <p>As an example I created 3 test tables, by using the population generator from the testdata_ninja package:</p> <p><pre>create table t_us as select * from table(testdata_ninja.population('US', 0.00001)); create table t_dk as select * from table(testdata_ninja.population('DK', 0.00001)); create table t_cn as select * from table(testdata_ninja.population('CN', 0.00001)); </pre></p> Random fun 409B1D9F2082B622E050007F01005D7B Sun, 06 Nov 2016 07:07:11 +0700 <p>It has been some fun evenings lately. I have added a lot of new functionality and new data domains to the <a href="">RANDOM_NINJA</a> package. One of the bigger updates is that I have created a Markov Chain function for text generation, so the random text is a lot more real looking:</p> <p><pre>SQL> select text_data.markov_sentence from dual; MARKOV_SENTENCE -------------------------------------------------------------------------------------------------------------- In but thruout the entire world various kinds of traps and snares as. SQL> select text_data.markov_sentence from dual; MARKOV_SENTENCE -------------------------------------------------------------------------------------------------------------- Safe however that not one-half of the best trappers use them. It requires. SQL> select text_data.markov_sentence from dual; MARKOV_SENTENCE -------------------------------------------------------------------------------------------------------------- The trappers use them. It is a humane way of killing. SQL> </pre></p> <p>This really adds a lot of functionality in terms of testing text functionality and text indexing, as the generated text is much closer to how real text would look.</p> <p>The person_random has received a lot of job related functionality, so we can now generate job titles, salaries and more:</p> <p><pre>SQL> select person_random.r_jobsector from dual; R_JOBSECTOR -------------------------------------------------------------------------------------------------------------- Educational Services SQL> select person_random.r_jobtitle from dual; R_JOBTITLE -------------------------------------------------------------------------------------------------------------- Consultant SQL> select person_random.r_jobtitle(r_jobsector => 'Educational Services') from dual; PERSON_RANDOM.R_JOBTITLE(R_JOBSECTOR=>'EDUCATIONALSERVICES') -------------------------------------------------------------------------------------------------------------- Teacher SQL> select person_random.r_salary(r_jobsector => 'Educational Services') from dual; PERSON_RANDOM.R_SALARY(R_JOBSECTOR=>'EDUCATIONALSERVICES') ---------------------------------------------------------- 94260 SQL> r 1* select person_random.r_salary(r_jobsector => 'Educational Services') from dual PERSON_RANDOM.R_SALARY(R_JOBSECTOR=>'EDUCATIONALSERVICES') ---------------------------------------------------------- 50400 SQL> select person_random.r_jobsector from dual; R_JOBSECTOR -------------------------------------------------------------------------------------------------------------- Management SQL> </pre></p> <p>The finance_random package has been updated with features related to accounts and account data generation. So we can generate random IBAN and BBAN valid bank accounts for a lot of countries, and we can generate random transaction types and account balances:</p> <p><pre>SQL> select finance_random.r_accountnumber from dual; R_ACCOUNTNUMBER -------------------------------------------------------------------------------------------------------------- FO1361785488259274 SQL> select finance_random.r_accountnumber('DK') from dual; FINANCE_RANDOM.R_ACCOUNTNUMBER('DK') -------------------------------------------------------------------------------------------------------------- DK1848327576613652 SQL> select finance_random.r_accounttype from dual; R_ACCOUNTTYPE -------------------------------------------------------------------------------------------------------------- Investment Account SQL> r 1* select finance_random.r_accounttype from dual R_ACCOUNTTYPE -------------------------------------------------------------------------------------------------------------- Investment Account SQL> r 1* select finance_random.r_accounttype from dual R_ACCOUNTTYPE -------------------------------------------------------------------------------------------------------------- Savings Account SQL> select finance_random.r_accounttransaction('Savings Account') from dual; FINANCE_RANDOM.R_ACCOUNTTRANSACTION('SAVINGSACCOUNT') -------------------------------------------------------------------------------------------------------------- Deposit SQL> r 1* select finance_random.r_accounttransaction('Savings Account') from dual FINANCE_RANDOM.R_ACCOUNTTRANSACTION('SAVINGSACCOUNT') -------------------------------------------------------------------------------------------------------------- Deposit SQL> r 1* select finance_random.r_accounttransaction('Savings Account') from dual FINANCE_RANDOM.R_ACCOUNTTRANSACTION('SAVINGSACCOUNT') -------------------------------------------------------------------------------------------------------------- Withdrawal SQL> select finance_random.r_accounttransaction('Investment Account') from dual; FINANCE_RANDOM.R_ACCOUNTTRANSACTION('INVESTMENTACCOUNT') -------------------------------------------------------------------------------------------------------------- Dividend SQL> r 1* select finance_random.r_accounttransaction('Investment Account') from dual FINANCE_RANDOM.R_ACCOUNTTRANSACTION('INVESTMENTACCOUNT') -------------------------------------------------------------------------------------------------------------- Cap. Gains Short SQL> select finance_random.r_accountbalance from dual; R_ACCOUNTBALANCE ---------------- 287555.25 SQL> select finance_random.r_accountbalance('Savings Account') from dual; FINANCE_RANDOM.R_ACCOUNTBALANCE('SAVINGSACCOUNT') ------------------------------------------------- 430614.3 SQL> r 1* select finance_random.r_accountbalance('Savings Account') from dual FINANCE_RANDOM.R_ACCOUNTBALANCE('SAVINGSACCOUNT') ------------------------------------------------- 5411.24 SQL> </pre></p> Building Facebook or Germany .... 3FA734494D560E42E050007F01004259 Wed, 26 Oct 2016 09:08:36 +0700 <p>Whether it is for work, or we are just playing with a new idea back home, there comes a time when we need to create some test data for whatever it is we are testing. The biggest problem with creating test data, is creating some data, that actually looks like real data. Because we want "real" data, if we really want to test how our application works or performs. Things like index selectivity, histograms and much more inside oracle will all behave differently depending on what kind of data there are in your tables. So creating life like data, is important.</p><p>Luckily I have previously created a base library that generates random data within a lot of different data domains. The <a href="">RANDOM_NINJA</a> package can create all that data, but because of the required flexibility in that library, it might not be user friendly, if you just want to create some tables fast to test on. So I decided to create a library that is based on the random_ninja library but binds it together in a little more easy way to create data.</p><p>The library is based on generators. All generators are pipelined functions, making it easy to use in create table statements. All generators by default will return 10 rows, but that can be controlled by setting the testdata_ninja.g_default_generator_rows parameter.</p><p>In the future there will be more control options for the generators, apart from just the rowcount. The current available generators (and their fields) are:</p><p> <ul> <li><strong>people</strong> - Generate a table of people with basic information. <ul> <li><strong>Country</strong> - Country of the person.</li> <li><strong>Identification</strong> - The ID number of the person. Will be a valid ID based on the Country.</li> <li><strong>First name</strong> - First name. Will be country and gender specific if data is available in random_ninja.</li> <li><strong>Middle name</strong> - Possible middle name. Will be country and gender specific if data is available in random_ninja.</li> <li><strong>Last name</strong> - Last name. Will be country and gender specific if data is available in random_ninja.</li> <li><strong>Birthdate</strong> - Birthdate of the person.</li> <li><strong>Gender</strong></li> </ul> </li> <li><strong>users</strong> - Generate a table of website users. <ul> <li><strong>Username</strong> - Username of website user.</li> <li><strong>Email</strong> - User email address.</li> <li><strong>Address1</strong> - First part of address. Always present.</li> <li><strong>Address2</strong> - Second part of address. Randomly present.</li> <li><strong>Zipcode</strong> - Zipcode of the address.</li> <li><strong>State</strong> - State of the address.</li> <li><strong>Creditcard</strong> - Type of credit card.</li> <li><strong>Creditcard number</strong> - Creditcard number.</li> <li><strong>Creditcard expiry</strong> - Creditcard expiry.</li> <li><strong>Password</strong> - The hashed password.</li> </ul> </li> <li><strong>cdr</strong> - Generate a table of cdr records. <ul> <li><strong>Orig imsi</strong> - IMSI number of the originating caller phone.</li> <li><strong>Orig isdn</strong> - Phone number of originating caller.</li> <li><strong>Orig imei</strong> - IMEI of originating caller.</li> <li><strong>Call type</strong> - Call type.</li> <li><strong>Call type service</strong> - Call service type.</li> <li><strong>Call start latitude</strong> - Latitude of the originator at the start of the call.</li> <li><strong>Call start longtitude</strong> - Longtitude of the orginator at the start of the call.</li> <li><strong>Call date</strong> - Date of the call.</li> <li><strong>Call duration</strong> - Call length in seconds.</li> <li><strong>Dest imsi</strong> - IMSI number of the destination caller phone.</li> <li><strong>Dest isdn</strong> - Phone number of destination phone.</li> <li><strong>Dest imei</strong> - IMEI of destination.</li> <li><strong>Network operator</strong> - Network operator code.</li> </ul> </li> <li><strong>articles</strong> - Create a table of articles. <ul> <li><strong>Author</strong> - The author of the article.</li> <li><strong>Written</strong> - Date article is written.</li> <li><strong>Headline</strong> - Headline.</li> <li><strong>Lead paragraph</strong> - Leading paragraph.</li> <li><strong>Main article</strong> - The main part of the article.</li> <li><strong>End paragraph</strong> - Final articel paragrpah.</li> </ul> </li> </ul> </p> Lots of random data 3F6AF888A63D241CE050007F01007AF3 Mon, 24 Oct 2016 09:07:18 +0700 <p>After a couple of long weekends, I have finally reached what I think is a good first attempt of a random data generation package. The number of different domains that you can generate data for, should be enough to cover most general use cases.</p> <p>The package, or packages, are divided into a couple of subcategories to make it a bit more easy to get an overview of the different capabilities. You can see the below list of what you can do with this:</p> <p> <ul> <li><strong>core_random</strong> - The basic random functionalities required. <ul> <li><strong>r_bool</strong> - Random true/false values. Chance of true can be specified.</li> <li><strong>r_natural</strong> - Random natural numbers. Range can be specified.</li> <li><strong>r_character</strong> - Random character. Casing, Choice pool and Symbols can be specified.</li> <li><strong>r_float</strong> - Random float number. Range and precision can be specified.</li> <li><strong>r_integer</strong> - Random integer number. Range can be specified.</li> <li><strong>r_string</strong> - Random string. Length and character pool can be defined.</li> <li><strong>r_hex</strong> - Random hex string. Length can be defined.</li> </ul> </li> <li><strong>text_random</strong> - Generate random pieces of text, to simulate comments, descriptions, articles and more. <ul> <li><strong>r_syllable</strong> - Random syllables.</li> <li><strong>r_word</strong> - Random word. Number of syllables, length and capitalizon can be specified.</li> <li><strong>r_sentence</strong> - Random sentence. The number of words in the sentence can be specified.</li> <li><strong>r_paragraph</strong> - Random paragraph. Number of sentences can be specified.</li> </ul> </li> <li><strong>time_random</strong> - Generate random time and date components. <ul> <li><strong>r_second</strong> - Random second.</li> <li><strong>r_minute</strong> - Random minute.</li> <li><strong>r_hour</strong> - Random hour. am/pm format can be specified.</li> <li><strong>r_ampm</strong> - Random am or pm.</li> <li><strong>r_millisecond</strong> - Random millisecond.</li> <li><strong>r_year</strong> - Random year. Min and max can be specified, as well as two digits result if needed.</li> <li><strong>r_month</strong> - Random month. Season can be specified.</li> <li><strong>r_day</strong> - Random day. Month can be specified.</li> <li><strong>r_date</strong> - Random date. Any component of the date format can be fixed if defined.</li> <li><strong>r_epoch</strong> - Random epoch date.</li> <li><strong>r_timestamp</strong> - Random timestamp. Any component of the timestamp can be fixed if defined.</li> </ul> </li> <li><strong>location_random</strong> - A wide variety of random location related data. <ul> <li><strong>r_country</strong> - Random country. Either full name, or short ISO form can be specified.</li> <li><strong>r_height</strong> - Random height. Result in feet and livable height can be specified.</li> <li><strong>r_depth</strong> - Random depth. Result in feet, and depth on land and not ocean can be specified.</li> <li><strong>r_latitude</strong> - Random latitude angle. Precision can be specified.</li> <li><strong>r_longtitude</strong> - Random longtitude. Precision can be specified.</li> <li><strong>r_coordinate</strong> - Random coordinates as a pair of latitude and longtitude points.</li> <li><strong>r_altitude</strong> - Random altitude, from 0 to upper atmosphere. Result in feet can be specified.</li> <li><strong>r_state</strong> - Random state name. Shortform option available. Country can also be specified.</li> <li><strong>r_zipcode</strong> - Random zip code. Country can specified for country specfic formats.</li> <li><strong>r_street</strong> - Random street name. Country and shortform can be specified.</li> <li><strong>r_address</strong> - Random address. Country and max numbering of street location can be specified.</li> <li><strong>r_city</strong> - Random city name. Country and state can be specified.</li> <li><strong>r_address2</strong> - Random address 2 line, such as apartment or name of building. Country can be specified.</li> <li><strong>r_level</strong> - Random building level.</li> </ul> </li> <li><strong>person_random</strong> - Random personal related data. <ul> <li><strong>r_age</strong> - Random age. Type (child, teen, adult, senior) can be specified.</li> <li><strong>r_birthday</strong> - Random birthday date. As with age, type can be specified.</li> <li><strong>r_firstname</strong> - Random first name. Both country and gender can be specified.</li> <li><strong>r_middlename</strong> - Random middle name. Country and gender can be specified.</li> <li><strong>r_lastname</strong> - Random last name. Country and gender can be specified</li> <li><strong>r_name</strong> - Random name. Country, Gender, Include middlename, Middle initial and prefix can be specified.</li> <li><strong>r_gender</strong> - Random gender.</li> <li><strong>r_prefix</strong> - Random prefix. Gender and country can be specified.</li> <li><strong>r_identification</strong> - Random valid identification number, based on the country specified.</li> <li><strong>r_suffix</strong> - Random suffix. Country and shortform can be specified.</li> </ul> </li> <li><strong>phone_random</strong> - Generate random data related to phones. <ul> <li><strong>r_imei</strong> - Random <u><i>valid</i></u> IMEI number. IMEISV version can be specified.</li> <li><strong>r_meid</strong> - Random <u><i>valid</i></u> MEID number.</li> <li><strong>r_country_calling_code</strong> - Random <u><i>valid</i></u> international code. + or 00 can be specified.</li> <li><strong>r_phonenumber</strong> - Random <u><i>valid</i></u> phone number. Generates <u><i>valid</i></u> landline or mobile numbers. Country and country code can be specified.</li> <li><strong>r_mcc</strong> - Random <u><i>valid</i></u> MCC code. Country can be specified.</li> <li><strong>r_mnc</strong> - Random <u><i>valid</i></u> MNC code. Country can be specified.</li> <li><strong>r_imsi</strong> - Random <u><i>valid</i></u> IMSI number. Country can be specified.</li> <li><strong>r_call_type</strong> - Random call type, for CDR data.</li> <li><strong>r_call_type_service</strong> - Random call type service, for CDR data.</li> <li><strong>r_operator_code</strong> - Random and <u><i>valid</i></u> operator code (pmnl) for CDR records. Country can be specified.</li> </ul> </li> <li><strong>finance_random</strong> - Generate random financial data. <ul> <li><strong>r_currency</strong> - Random <u><i>valid</i></u> ISO currency. ISO shortform can be specified.</li> <li><strong>r_currencypair</strong> - Random base and quote currencies. Both can be specified.</li> <li><strong>r_creditcard</strong> - Random <u><i>valid</i></u> credit card name. Shortform can be specified.</li> <li><strong>r_creditcardnum</strong> - Random <u><i>valid</i></u> creditcard number. Creditcard type can be specified.</li> <li><strong>r_expirydate</strong> - Random credit card expiry date. Future data can be specified.</li> <li><strong>r_amount</strong> - Random amount. Range and precision can be specified.</li> </ul> </li> <li><strong>web_random</strong> - Generate random web/internet related data. <ul> <li><strong>r_tld</strong> - Random top level domain. Generic and country inclusion can be specified.</li> <li><strong>r_domain</strong> - Random domain name. TLD can be specified.</li> <li><strong>r_email</strong> - Random email address. Specify if you want to use real name.</li> <li><strong>r_ipv4</strong> - Random <u><i>valid</i></u> ipv4 ip address.</li> <li><strong>r_url</strong> - Random URL. Specify if you want to use https and if you want to add query parameter.</li> <li><strong>r_ipv6</strong> - Random <u><i>valid</i></u> ipv6 ip address.</li> </ul> </li> <li><strong>consumer_random</strong> - Generate random consumer related data. <ul> <li><strong>r_category</strong> - Random consumer category. You can choose between all, food ro non-food categories.</li> <li><strong>r_food_item</strong> - Random food item. Category can be specified.</li> <li><strong>r_nonfood_item</strong> - Random non-food item. Category can be specified.</li> <li><strong>r_service_item</strong> - Random service related item. Category can be specified.</li> </ul> </li> </ul> </p> <p>Phew... no wonder I felt like this was taking a lot of time to create. I really hope that people can use this for something. I know that there are several times where I have to create random test data, and I always end up using excel, or even worse, just one row with 123,123,123,asdf,asdf like data.</p><p>Now we have a better option</p> Generating random data 3DCC872FC83EDD55E050007F01001568 Sat, 01 Oct 2016 23:40:36 +0700 <p>We all need to create random data every now and again. Most of the times I have resolved to simply using <a href="">dbms_random</a> to generate random strings and numbers. The problem with approach though, is that data does not look like "real" data, and indexing will not be realistic. The Alexandria library contains a package that has a few functions <a href="">random_util_pkg</a>, but it is limited in the type of data you can create, and I felt that there were a couple of more parameters where I needed a bit more control on the output.</p> <p>So I have created <a href="">RANDOM_NINJA</a> which for now has the basic functions that I needed.</p><p>Just see the quick demo I have created below:</p> NPG - The package manager for Oracle PL/SQL 3B756949B69ADE86E050007F01007BA1 Thu, 01 Sep 2016 12:23:11 +0700 I finally got the most rough edges done now. After a long time of trying to find some spare time during the evenings, I have completed what I think, has been a great feature missing from PL/SQL for a long long time: A real package manager. I've always believed that PL/SQL has a great community and that there exists a lot of packages out there, that deserves a broader audience. Not only that, but it needs to be easier to share and install these community packages.<p>Say welcome to NPG, an oracle package manager for PL/SQL. It supports most of the basic features that package managers from other languages does (NPM, RubyGems, PIP etc) and I will add more and more functionality to it, over the coming weeks.</p> Building something new and fun for pl/sql F96FBE6EA0254CE2B1B9BC7C1E7C5C80 Mon, 01 Aug 2016 10:56:37 +0700 <p>So having done a lot coding in different languages (recently python, ruby and javascript) I came to the conclusion that pl/sql is missing an important functionality or addition if you like.</p> <p>So for a while now, whenever I've had some spare time (a bit difficult with 3 small kids) I've been coding up a fun little project.</p> <p>It is now pretty close to beta testing, and so this week I will finishing of the rough edges and bring a couple of small teasers as well</p> JWT - JSON Web Tokens C4E67168AF3340499E7521DBC00F502C Sat, 18 Jun 2016 08:53:42 +0700 <p>After having worked on a couple of SAML issues, trying to debug some encryption errors, I was getting really tired of really big SAML messages.</p> <p>So I started looking around for alternatives, and found the <a href="">JWT</a> website. A lot more straight forward and plenty secure for what most people are looking to do with security tokens.</p> <p>To quote from the website: </p> <p> JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the <strong>HMAC</strong> algorithm) or a public/private key pair using <strong>RSA</strong>.</p> <ul> <li><strong>Compact:</strong> Because of its smaller size, JWTs can be sent through an URL, POST parameter, or inside an HTTP header. Additionally, the smaller size means transmission is fast.</li> <li><strong>Self-contained:</strong> The payload contains all the required information about the user, avoiding the need to query the database more than once.</li> </ul> T minus 10 ... 9 ... 8 ... 7 568CD170401A41E6A0EF63938F3814AA Thu, 31 Dec 2015 06:51:20 +0700 <p>While creating the <a href="/code_is_good/dev_qa_prod.assert?condition=codemonth:::12:">lightweight monitoring package</a>, I could see that I could use the same technique for another little project I wanted to create. A fast instance wide countdown, that was not dependent on a table or a sequence. Since I already had the basic framework ready, it was pretty fast to implement.</p> Exponential Moving Averages, Histograms and Gauges 66E32B07FDE0433693C79E9010078CCB Sun, 13 Dec 2015 17:20:16 +0700 <p>Had a couple of extra minutes today, so I thought about if I could add something to the lightweight monitoring that I created earlier last month. One of things I really wanted, was to add exponential moving average. Moving averages, are normally a little better indicator of the overall performance, as a normal average tends to hide momentary bumps, that you might want to investigate.</p><p>Another thing that could be useful, was a histogram over the call times. A histogram, is good way of showing the distribution of calls, which will also show us, the amount of outliers that are there. Outliers can be hidden in averages, so having something to flag this, is a good thing.</p> PLSQL, syslog and the story of Bolas spiders A68700AC16814650923E5F8F75577C14 Wed, 09 Dec 2015 16:19:59 +0700 <p>So I am setting up a server for a bigger project that I'm playing with at the moment, and I wanted a quick and easy correlated way of logging messages/errors locally on the machine as well as using my airbrake utility. So I wanted all my components to write to the syslog on the linux server I'm setting up, even my database.</p><p>So the very first thing you want to read, whenever you want to implement any protocol/service/functionality, is the RFC if one exists. Syslog RFC number is 3164.</p> Semantic versioning - Verifying full version ranges 498BE3DD70824CD09484C19628690C7E Sun, 29 Nov 2015 06:05:26 +0700 <p>The last added function was the cmp function, which we needed to finish the overall goal, of the package: To create a "satisfies" function, that can verify if a specific version is satisfied within a range of versions.</p> <p>The different ranges, that can be specified are X-ranges (Wildcard, i.e. 1.2.* or 2.*), Tilde-ranges (minimum requirements for minor versions, ~1.2.3, which will allow versions between 1.2.3 and up to 1.2.n, but not 1.3.0) and Caret-ranges and Hyphen ranges. For a full description and detailed example, you should read the following 2 articles:</p>