@bytesnz

Jack Farley, Computer Engineer

  • home
  • about
  • 6/16/2022

    JS Binary Decoder

    software

    I have recently been dealing with parsing binary data packets from various sources, and have published two NPM libraries from it - binary-decoder and sbd-direct-ip. How's how they came to be.

  • 4/19/2022

    New Zealand Low Power Wide Area Networks

    electronicssoftware

    I have started experimenting with low power wide area networks (LP-WAN) in New Zealand using a u-blox SARA-R410-02B. Here is is what I have experimented with so far.

  • 2/7/2022

    Starting with ESP32

    esp32electronicssoftwaretrap-watch

    I started developing my trap-watch project on an ESP32-CAM using the ESP IDF. Here is the newbie difficulties I ran into.

  • 2/4/2022

    VIM Window Rabbit Hole

    linux

    Developing ESP-IDF components I thought it would be great if I could make a command to open all the files for a component at once. What a rabbit hole it was. Here is how I did it.

  • 1/28/2022

    Parsing Arguments in Bash

    linux

    Making a script parse arugments in Bash took me way too long last time I did it, so here is a nice full example of how to do it using getopt

  • 1/8/2022

    Where's my bait gone?

    electronicsconservationtrap-watch

    I have recently started trapping some introduced predators around my local area and have had baits and pre-feed disappear with nothing to show for it, so I decided to get sparky and see if I could catch the culprit in the act.

  • 3/26/2021

    Bathymetric Contours

    gisgeoserveroceans

    Finally got around to creating a SLD style for GeoServer to display bathymetric contour lines using the GEBCO gridded bathymetric data. Here's how.

  • 2/25/2021

    shm That Cache

    yarndockerdevelopment

    I am often trying to find more space on my hard drives and found today my own docker containers wasting space thanks to ! Here's how I fixed it.

  • 2/2/2021

    Blob blob blob blob Stayin' Alive Stayin' Alive

    softwarejavascriptdebuggingmemory

    Upon recently trying Deezer again, I found their web app ate all my memory when running in Firefox, so I decided to see if I could find out why. I got as far as memory-file-data/string and Blobs. Here's how.

  • 12/25/2020

    A new Litter Survey

    softwarelitterlitter-survey

    In a culmination of litter surveys and litter picks, linked data and data exploration, and remoteStorage and ActivityPub, I have created a web-based litter pick/survey app that I hope will allow federated citizen science.

  • 12/13/2020

    Hoe Stream Trash

    litterpi-trash-camlitter-pick

    My latest litter pick target was Hoe Stream and the White Rose Lane Local Nature Reserve. Here's how it went.

  • 11/24/2020

    Release the Beast

    softwaregitlabcirelease

    I just created a Gitlab CI job to create a release with information from a CHANGELOG.md file for some of my projects. Here's how I did it.

  • 9/30/2020

    Gitlab CI Caching

    gitlabcontinuous integration

    I noticed something strange happening during build process during a multi-tasking bug fix. Turns out I was using Gitlab CI's caching incorrectly. I should have been using artifacts. Here's what I saw.

  • 9/21/2020

    Birthday Trash

    litterpi-trash-cam

    As a birthday treat, I took the day off work to try out my electronerised litter picker. Here's how it went.

  • 9/13/2020

    Pi Trash Cam

    raspberry pielectronicslitterpi-trash-cam

    In preparation for a day of litter picking, I finally got round to a project idea - attaching a camera to a litter picker to record it all. Here's what I did.

  • 8/25/2020

    yarn add --dev webdriverio

    first-draftsoftwaretestinggitlab

    I finally started implementing UI testing on first-draft using WebdriverIO. While writing tests was easy, getting the tests running was a little more difficult. Here is how I did it.

  • 8/16/2020

    Hooray! My new blog is live! Based on Sapper, using MongoDB and eventually ActivityPub and ActivityStreams, it will be my federated posting hub to the world.

    softwareblog
  • 8/9/2020

    Removing EXIF Data from Photos

    exifphotos

    Creating this new blog, I wanted to make sure there was no metadata data leaking personal information. Here's how I removed all the metadata tags except the ones I wanted from my photos.

  • 8/9/2020

    tmux List and Reattach

    tmuxbashsoftware

    Using tmux for your terminal multiplexer but want an easy to reattach to a session? Here's a small bash script to do it.

  • 8/8/2020

    Selectable Shell Examples

    markdownsyntaxcss

    Here's how to help your readers save time by making your post's shell commands easy to select and copy - with a simple CSS property.

  • 8/5/2020

    Be Dates You

    mongodbdocker

    Making my new blog, I didn't initially set the published dates to be native dates in the database. Here what I did to change them ...and do all the upgrades I needed.

    For this new blog, I am using MongoDB for storing all the articles and notes. During development, when I initially copied my old articles over, the code wasn't set up to store the published dates as a native BSON date.

    Once I had fixed the code to store the dates as native dates, I needed to change the dates stored as strings to native dates for the sorting to work correctly.

    Digging into the MongoDB documentation, I found the $dateFromString aggregator that I could use with an update aggregation pipeline.

    After testing in the very helpful MongoDB Web Shells playpen that are included in the documentation, I came up with the following command to run.

    db.objects.updateMany({ published: { $type: "string" } }, [ { $set: { published: { $dateFromString: { dateString: "$published" } } } } ] );

    The command:

    1. filters the articles in the objects collection, leaving only those with a string type value for the published date
    2. sets the published date to be a native date using the existing value as the string for the $dateFromString operator.

    I prepared myself and adjusted the command for running it on my data. Command in hand, I jumped onto the mongo shell, ran the command and...

    > db.objects.updateMany({ published: { $type: "string" } }, [ { $set: { published: { $dateFromString: { dateString: "$published" } } } } ] );
    2020-08-05T08:38:49.463+0100 E QUERY    [thread1] Error: the update operation document must contain atomic operators :
    DBCollection.prototype.updateMany@src/mongo/shell/crud_api.js:625:1
    @(shell):1:1
    > 
    Uh oh. What?! Huh?! But...

    I went back and made sure that I had the command right, which I did. Then I wondered... what version am I running...

    Turns out it was the latest and greatest... available through the standard repos - v3.6, whereas the latest stable version was v4.4. Sad face.

    Not wanting to deal with extra faff, I decided to use docker for running Mongo instead of installing it. Brimming with confidence, I jumped straight for the new version and tried to mount my data from v3.6

    $ docker run --name mongodb -v /var/lib/mongodb:/data/db -p 27017:27017 mongo
    ...
    {"t":{"$date":"2020-08-05T17:49:08.027+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=5415M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
    {"t":{"$date":"2020-08-05T17:49:08.753+00:00"},"s":"W",  "c":"STORAGE",  "id":22347,   "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
    {"t":{"$date":"2020-08-05T17:49:08.753+00:00"},"s":"F",  "c":"STORAGE",  "id":28595,   "ctx":"initandlisten","msg":"Terminating.","attr":{"reason":"95: Operation not supported"}}
    {"t":{"$date":"2020-08-05T17:49:08.753+00:00"},"s":"F",  "c":"-",        "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":1100}}
    {"t":{"$date":"2020-08-05T17:49:08.753+00:00"},"s":"F",  "c":"-",        "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
    Doh.

    I would to upgrade from v3.6 to v4.0, to v4.2, to v4.4. Because I was using docker it was super easy.

    ## Start the mongodb docker container, ensuring it will delete itself when I stop it
    $ docker run --rm -d --name mongodb -v /var/lib/mongodb:/data/db -p 27017:27017 mongo:4.0
    xxxxxxxxxxxxxxxxxxxxx
    ## Run the magic upgrade command
    $ docker exec -it mongodb mongo --eval 'db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } )'
    MongoDB shell version v4.0.19
    connecting to: mongodb://127.0.0.1:27017/?gssapiServiceName=mongodb
    Implicit session: session { "id" : UUID("xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx") }
    MongoDB server version: 4.0.19
    { "ok" : 1 }
    $ docker stop mongodb

    then rinse and repeat for 4.2 and 4.4. Once upgraded, I was able to run the command. Yay!

    > db.objects.updateMany({ published: { $type: "string" } }, [ { $set: { published: { $dateFromString: { dateString: "$published" } } } } ] );
    { "acknowledged" : true, "matchedCount" : 35, "modifiedCount" : 35 }
    Close
  • 7/8/2020

    Testing vue components

    softwarevuetestingjavascript

    I recently needed to test that some Vue components were creating the correct HTML. To do this, I decided to create snapshots of Object representations of the rendered HTML.

  • 2/20/2019

    No More Numbers

    javascripthtml5software

    HTML5 number inputs aren't useful, but tel inputs, have all the power

  • 12/21/2018

    A Hacker "Hacked" Me

    securityemailgrep

    I decided to look into the extortion emails I have been getting and wrote a small script to extract the bitcoin addresses that have been used.

  • 12/7/2018

    Mouse Surgery

    pledge to not upgradeelectronics

    As part of my pledge not to upgrade, I decided to repair two of my failing mice instead of replacing them with a brand new model (as tempting as it was). Here's what I did.

  • 10/29/2018

    Danger Danger, Highly Reactive

    vuereactivegraphingdata
  • 9/11/2018

    Switching to SSR

    reactssrreact-loader
  • 8/10/2018

    Clean Docker Registry

    nodedockerconsolejavascript
  • 2/1/2018

    Testing on the Filesystem

    testingjavascript
  • 1/26/2018

    Tag You Are

    javascripttags
  • 1/25/2018

    NaN Got Me

    rubber duck savejavascriptnodejsjson
  • 4/28/2017

    All our app's tabs are belong to us

    angularjavascriptlocalstoragerxjssessionstorage
  • 2/20/2016

    Developing NPM modules

    nodejsnpm
  • 2/20/2016

    Nfa + Nfb or N(f+fa+fb)

    arrayfunctionjavascriptjsperf
  • 10/31/2015

    Photo Layout

    arrangercssgallery hierarchyhtmljavascriptphotostinymcewordpresssoftware
  • 10/31/2015

    Browning Pass HideAway Web Site

    cssgallery hierarchyhtmlwordpress
  • 10/17/2015

    Think Mobile

    mobileweb
  • 10/13/2015

    Pledge to Refuse and Not Buy Bottled Water

    environmentpledges
  • 2/25/2015

    Run PHP run!

    maximum execution timephptrial and error
  • 11/26/2014

    Gallery Hierarchy

    galleryhierarchyphotos
  • 10/3/2014

    Vim and functions

    functionnavigationtagsvim
  • 9/26/2014

    I'm making hierarchies

    hierarchical datahierarchyphp
  • 9/4/2014

    Google Sheets fun

    googlegoogle scriptgoogle sheetsjavascriptsheets
  • 8/13/2014

    Travel Photos

    photo managementphotostravel photos
  • 8/13/2014

    Image managment scripts

    photo managementphotos
  • 7/31/2014

    MeldCE logo

    csshtml5javascriptsvg
  • 4/26/2014

    MoltenDB

    databasemoltendbmongodbnodejs
  • 10/29/2013

    The Expensive Side

    house
  • 9/22/2013

    abcde

    music
  • 7/16/2013

    Energizer Power Plus Rechargeable Batteries

    gadgets
  • 3/4/2013

    USAR FOGTeX

    fogtex
  • 2/21/2013

    Personal Gear Bag

    rescuegear
  • 2/18/2013

    Geocaching Stamp

    stampgeocaching
  • 4/2/2007

    Call Record Presenter

    parserpythonsqliteweb
  • 9/25/2004

    Kimi Ora School

    csshtml
  • 6/25/2004

    All About Catering Web Site

    csshtml
  • 7/26/2003

    Quantum Accounting Web Site

    csshtml
Copyright BytesNZ 2022. Powered by Sapper
[matrix]