How to use Google Sheet to get bulk page status check and email alerts

If your website have thousands of page and you are manually checking them for latest page status, page speed or want to get alert when page is down, Google sheet will be useful.

Core concept is simple. Using macro, you can create custom functions to fetch webpage and check for status. Using Gmail, you can send the update. Lets start this process.

Step 1:

Make a copy of this sheet or create a blank sheet. Mark a column for full URL.

Step 2: Go to tools > script editor

Step 3: Copy this script

function statuscode(url) {
return UrlFetchApp.fetch(url).getResponseCode()
}

Save the code. Now you can use formula like this. =statuscode(” https://www.servloci.com/how-to-use-google-sheet-to-get-bulk-page-status-check-and-email-alerts/”)

Flask Bootstrap quick reference

installation :

pip install flask-bootstrap  

This will install bootstrap version 3.3.7

Import Bootstrap into Flask Application

from flask_bootstrap import Bootstrap
Bootstrap(app)

Extend Basic Template

{% extends "bootstrap/base.html" %}

Available building blocks

Available blocks

Block nameOuter BlockPurpose
doc Outermost block.
htmldocContains the complete content of the <html> tag.
html_attribsdocAttributes for the HTML tag.
headdocContains the complete content of the <head> tag.
bodydocContains the complete content of the <body> tag.
body_attribsbodyAttributes for the Body Tag.
titleheadContains the complete content of the <title> tag.
stylesheadContains all CSS style <link> tags inside head.
metasheadContains all <meta> tags inside head.
navbarbodyAn empty block directly above content.
contentbodyConvenience block inside the body. Put stuff here.
scriptsbodyContains all <script> tags at the end of the body.

Usages of all blocks are self-explanatory.

Support for Navigation using Flask-nav

Support for Forms using Flask-wtf
{% import “bootstrap/wtf.html” as wtf %}

How to add a specific folder to Pythonpath and why?

pythonpath is specific bash variable used by python interpreteur and compiler to search for modules. Most of time, we need to re-use codes and for it , specific function with desired set of input/output is required.

If you need to send different type of mails( from crontasks, from celery , after login , after invocation of reports api etc) , you can simply import send_email function and reuse it without re-writing email script (import email mode, attach file, fill out sender and reciepient address). For it, neither you have to engage a alltime running api service with all functions active.

Store all scripts in separate folder and add them to pythonpath. Next time, python can find and execute them.

Functions could be rearranged according to need. Now come to main point.

For Ubuntu

edit your .bashrc file and add this line.
export PYTHONPATH=$PYTHONPATH:/mnt/sdb1/mylibs

In this example, my personal library is on another disk and is in mylibs folder.

Hope , this will make your life better.

A connection to the notebook server could not be established. The notebook will continue trying to reconnect, but until it does, you will NOT be able to run code. Check your network connection or notebook server configuration

This problem is due to incorrect configuration of torando server. You need to uninstall tornado version 6 and install tornado version 5.1.1

To uninstall run : pip uninstall tornado

Install tonado : pip install tornado==5.1.1

How and why use SQLite in R

If you need small amount of data to work with( like development of a alert system to store alert conditions). Using a full fleged db would be painful.

Most of Programming system have sqlite drivers inbuilt, therefore no need to install any other driver.

First you need to install R package for sqlite database

install.packages("RSQLite") 

Basic usage of RSQLite package ( code snippet)

> library(DBI) 
> con <- dbConnect(RSQLite::SQLite(), ":memory:")
> con2 <- dbConnect(RSQLite::SQLite(), "mydb.sqlite")
> dbListTables(con) #list the tables character(0)
> dbWriteTable(con, "mtcars", mtcars)
> dbListTables(con) #list the tables
> res <- dbSendQuery(con, "SELECT * FROM mtcars WHERE cyl = 4")
> dbFetch(res)
> dbClearResult(res)

R package Installation failed: Timeout was reached: Resolving timed out after 10000 milliseconds

If you are facing problems like above, timeout may be problem of your network( domain name resolution). 
Possible solution is to update the network dns . It should be fast dns like Google DNS or Cloudflare DNS 

Google DNS – 8.8.8.8,8.8.4.4
Cloudflare DNS – 1.1.1.1

With low latency network and responsive dns, this problem will be resolve as resolved in above image. 

This post is also answer to how to install IRkernal for jupyter notebook 

Flask sqlalchemy error : SQLALCHEMY_TRACK_MODIFICATIONS

Problem :FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
‘SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and ‘

Solution:  app.config[‘SQLALCHEMY_TRACK_MODIFICATIONS’] = False

Why you need structured data for better SEO ?

Webpage is not 100% data unlike other data only formats ( excel sheets, json , xml , csv ) etc . It has other goal – present information in visual manner, therefore data is mixed with lot of unwanted content( tags ). 

Each and every webpage have some unique design guidelines to present information and carry data. Crawler are not so smart to fetch all information from webpage. Beside fetching data, it is not easy to catagorize and index data correctly all the time. Therefore, there must be some unique method to carry data , without worrying about the html of webpage. 

Structured data comes to rescue. It follow a fixed , validated semantics to present data. I will recommed to use structured data using javascript ( JSON format)

You can find more about it from Google search gallery

In search gallry , you will find information about webpage component and its structure . 

Vue JS Issue – Devtool inspection is not available

Full description of problem:  Vue.js is detected on this page. Devtools inspection is not available because it’s in production mode or explicitly disabled by the author.

Cause: You may be using compressed production version of vue

Solution : You need to use development version with debugging support.
<script src=”https://vuejs.org/js/vue.js” ></script>



vue.js source

If your problem solved, share this to others.