@bytesnz

Jack Farley, Computer Engineer

  • home
  • about
  • 12/25/2020, 12:13:48 AM

    A new Litter Survey

    softwarelitterlitter-survey

    In a culmination of litter surveys and litter picks, linked data() and data exploration, and remoteStorage and ActivityPub, I have created a web-based litter pick/survey app that I hope will allow federated citizen science.

  • 12/13/2020, 6:00:00 PM

    Hoe Stream Trash

    litterpi-trash-camlitter-pick

    My latest litter pick target was Hoe Stream and the White Rose Lane Local Nature Reserve. Here's how it went.

  • 11/24/2020, 6:00:00 PM

    Release the Beast

    softwaregitlabcirelease

    I just created a Gitlab CI job to create a release with information from a CHANGELOG.md file for some of my projects. Here's how I did it.

  • 9/30/2020, 9:00:00 PM

    Gitlab CI Caching

    gitlabcontinuous integration

    I noticed something strange happening during build process during a multi-tasking bug fix. Turns out I was using Gitlab CI's caching incorrectly. I should have been using artifacts. Here's what I saw.

  • 9/21/2020, 6:00:00 PM

    Birthday Trash

    litterpi-trash-cam

    As a birthday treat, I took the day off work to try out my electronerised litter picker. Here's how it went.

  • 9/13/2020, 12:12:13 AM

    Pi Trash Cam

    raspberry pielectronicslitterpi-trash-cam

    In preparation for a day of litter picking, I finally got round to a project idea - attaching a camera to a litter picker to record it all. Here's what I did.

  • 8/25/2020, 9:29:58 PM

    yarn add --dev webdriverio

    first-draftsoftwaretestinggitlab

    I finally started implementing UI testing on first-draft using WebdriverIO. While writing tests was easy, getting the tests running was a little more difficult. Here is how I did it.

  • 8/16/2020, 7:04:44 PM

    Hooray! My new blog is live! Based on Sapper, using MongoDB and eventually ActivityPub and ActivityStreams, it will be my federated posting hub to the world.

    softwareblog
  • 8/9/2020, 5:40:49 PM

    Removing EXIF Data from Photos

    exifphotos

    Creating this new blog, I wanted to make sure there was no metadata data leaking personal information. Here's how I removed all the metadata tags except the ones I wanted from my photos.

  • 8/9/2020, 10:21:41 AM

    tmux List and Reattach

    tmuxbashsoftware

    Using tmux for your terminal multiplexer but want an easy to reattach to a session? Here's a small bash script to do it.

  • 8/8/2020, 2:58:48 PM

    Selectable Shell Examples

    markdownsyntaxcss

    Here's how to help your readers save time by making your post's shell commands easy to select and copy - with a simple CSS property.

  • 8/5/2020, 6:48:38 PM

    Be Dates You

    mongodbdocker

    Making my new blog, I didn't initially set the published dates to be native dates in the database. Here what I did to change them ...and do all the upgrades I needed.

  • 7/8/2020, 4:00:00 PM

    Testing vue components

    softwarevuetestingjavascript

    I recently needed to test that some Vue components were creating the correct HTML. To do this, I decided to create snapshots of Object representations of the rendered HTML.

  • 2/20/2019, 12:00:00 AM

    No More Numbers

    javascripthtml5software

    HTML5 number inputs aren't useful, but tel inputs, have all the power

  • 12/21/2018, 12:00:00 AM

    A Hacker "Hacked" Me

    securityemailgrep

    I decided to look into the extortion emails I have been getting and wrote a small script to extract the bitcoin addresses that have been used.

  • 12/7/2018, 12:00:00 AM

    Mouse Surgery

    pledge to not upgradeelectronics

    As part of my pledge not to upgrade, I decided to repair two of my failing mice instead of replacing them with a brand new model (as tempting as it was). Here's what I did.

  • 10/29/2018, 7:00:00 PM

    Danger Danger, Highly Reactive

    vuereactivegraphingdata
  • 9/11/2018, 7:00:00 AM

    Switching to SSR

    reactssrreact-loader
  • 8/10/2018, 4:00:00 PM

    Clean Docker Registry

    nodedockerconsolejavascript
  • 2/1/2018, 12:00:00 AM

    Testing on the Filesystem

    testingjavascript
  • 1/26/2018, 12:00:00 AM

    Tag You Are

    javascripttags
  • 1/25/2018, 9:00:00 PM

    NaN Got Me

    rubber duck savejavascriptnodejsjson
  • 4/28/2017, 10:25:34 AM

    All our app's tabs are belong to us

    angularjavascriptlocalstoragerxjssessionstorage
  • 2/20/2016, 2:33:19 AM

    Developing NPM modules

    nodejsnpm
  • 2/20/2016, 1:48:13 AM

    Nfa + Nfb or N(f+fa+fb)

    arrayfunctionjavascriptjsperf
  • 10/31/2015, 3:36:29 PM

    Photo Layout

    arrangercssgallery hierarchyhtmljavascriptphotostinymcewordpresssoftware
  • 10/31/2015, 2:33:49 PM

    Browning Pass HideAway Web Site

    cssgallery hierarchyhtmlwordpress
  • 10/17/2015, 4:04:57 PM

    Think Mobile

    mobileweb
  • 10/13/2015, 9:30:59 AM

    Pledge to Refuse and Not Buy Bottled Water

    environmentpledges
  • 2/25/2015, 7:42:33 AM

    Run PHP run!

    maximum execution timephptrial and error

    I recently changed from using Wordpress cron jobs to using disconnecting AJAX requests for scanning for images with Gallery Hierarchy to try and get a little more stable and reliable scanning system.

    One of the problems that didn't go away doing this was dealing with the maximum execution time of a PHP script. This is set by (among other things) PHP max_execution_time setting in the PHP configuration file (php.ini).

    After coming up with a solution to check if the scan job had been running for longer than this execution time and then restarting it and finding that it wasn't working, I did a couple of tests.

    My first test was this simple PHP script:

    <?php
      $file = 'test1.log';
      file_put_contents($file, "New test started at " . time() . "\n", FILE_APPEND);
      file_put_contents($file, "max_execution_time: " . ini_get('max_execution_time') . "\n", FILE_APPEND);
      file_put_contents($file, "max_input_time " . ini_get('max_input_time') . "\n", FILE_APPEND);
      $i = 0;
      $time = 0;
      while (1) {
        file_put_contents($file, "" . $i++ . "\n", FILE_APPEND);
        sleep(1);
      }
    ?>

    After running the script in the console while monitoring the test1.log file (and having it running for more that 400s to be expected as the max_execution_time is 0 (unlimited) in the console), I tried running it in the browser and got the following in test1.log:

    New test started at 1424797632
    max_execution_time: 300
    max_input_time 60
    0
    1
    ...
    300
    301
    302
    ...
    500
    ...

    After seeing this, I wondered if the maximum execution time was the maximum CPU time rather than just time, so I adjusted my test script to the following:

    <?php
      $file = 'test1.log';
      file_put_contents($file, "New test started at " . time() . "\n", FILE_APPEND);
      file_put_contents($file, "max_execution_time: " . ini_get('max_execution_time') . "\n", FILE_APPEND);
      file_put_contents($file, "max_input_time " . ini_get('max_input_time') . "\n", FILE_APPEND);
      $i = 0;
      $time = 0;
      while (1) {
        if (time() > $time) {
          file_put_contents($file, "" . $i++ . "\n", FILE_APPEND);
          $time = time();
        }
      }
    ?>

    After running it in the browser again, the output was a little more expected and confirmed my suspicions.

    New test started at 1424797725
    max_execution_time: 300
    max_input_time 60
    0
    1
    ...
    300
    301
    302
    ...
    371
    372

    Even though it didn't get killed bang on 300 seconds, it did get killed.

    So after the tests and finding that it judging whether something has been killed simply based on the time and the max_execution_time was a bad idea, I decided to change my approach. Instead of seeing how long it has been running for, I see how long it has been since its status was last saved - something that should happen every 10 seconds. If there hasn't been a status update in a while (30 seconds), I assume that the scan job has been killed and try and restart it. It should be noted that if it dies due to an error, it will not be restarted. I have also set the execution limit to 0 (using settimelimit) in the hope that it is on a server that it will work on.

    One function that may be of use for monitoring cpu time is getrusage, which can be used to monitor CPU time among a few other useful stats.

    Some other ways that I considered was:

    • using getrusage in the PHP script to its running time so that when it got close to its end, it could die gracefully/signal it was going to die. This would however add processing overhead.
    • storing the PID of the PHP script and then running once that process goes. This, on a super busy machine could lead to the missing the PHP script dying - if the server happened to have lots of PID reuse and a new process happened to pick up the PID of the PHP script and the run indefinitely... unlikely(ish), but it could happen.
    Close
  • 11/26/2014, 6:35:32 AM

    Gallery Hierarchy

    galleryhierarchyphotos
  • 10/3/2014, 7:22:04 AM

    Vim and functions

    functionnavigationtagsvim
  • 9/26/2014, 3:18:53 AM

    I'm making hierarchies

    hierarchical datahierarchyphp
  • 9/4/2014, 1:45:18 PM

    Google Sheets fun

    googlegoogle scriptgoogle sheetsjavascriptsheets
  • 8/13/2014, 3:14:22 AM

    Travel Photos

    photo managementphotostravel photos
  • 8/13/2014, 3:02:22 AM

    Image managment scripts

    photo managementphotos
  • 7/31/2014, 3:43:45 PM

    MeldCE logo

    csshtml5javascriptsvg
  • 4/26/2014, 6:34:09 AM

    MoltenDB

    databasemoltendbmongodbnodejs
  • 10/29/2013, 12:09:07 AM

    The Expensive Side

    house
  • 9/22/2013, 7:00:08 PM

    abcde

    music
  • 7/16/2013, 8:26:10 PM

    Energizer Power Plus Rechargeable Batteries

    gadgets
  • 3/4/2013, 6:27:40 PM

    USAR FOGTeX

    fogtex
  • 2/21/2013, 9:09:54 PM

    Personal Gear Bag

    rescuegear
  • 2/18/2013, 11:10:34 PM

    Geocaching Stamp

    stampgeocaching
  • 4/2/2007, 3:50:02 PM

    Call Record Presenter

    parserpythonsqliteweb
  • 9/25/2004, 9:48:10 AM

    Kimi Ora School

    csshtml
  • 6/25/2004, 8:53:53 AM

    All About Catering Web Site

    csshtml
  • 7/26/2003, 7:52:11 AM

    Quantum Accounting Web Site

    csshtml
Copyright BytesNZ 2021. Powered by Sapper
[matrix]