end0tknr's kipple - 新web写経開発

http://d.hatena.ne.jp/end0tknr/ から移転しました

install postgres 11.2 & postgis 2.5 & mapserver 7.2 (2019年版)

install postgres 9.6 & postgis 2.3 & mapserver & mapserver 7.0 - end0tknr's kipple - 新web写経開発

国土交通省にあるGISデータをPostGISへインポート(2017年版) - end0tknr's kipple - 新web写経開発

上記2エントリの2019年版。

面倒なので、前回との差異のみを以下に記載します。

PostGIS 2.5 は、PostgreSQL 11.2 にもinstall OK

https://postgis.net/docs/manual-2.5/postgis_installation.html の「Required」に「PostgreSQL 9.4 or higher.」と記載されています。

「ver.11.2って上げ過ぎかな?」とも思いましたが、 結果的に問題なく、PostGIS 2.5をinstallできました。

postgresのcreatelangコマンドが削除されていました

前回のエントリににて

$ /usr/local/pgsql/bin/createlang -U postgres plpgsql gis_test

を実行していますが、postgres 11.2ではcreatelangがありませんでした。

$ /usr/local/pgsql/bin/psql -U postgres gis_test
psql> CREATE LANGUAGE plpgsql;
ERROR:  language "plpgsql" already exists

の手順で操作すればよいようです。 ただし、この「CREATE LANGUAGE plpgsql」はそもそもPostGISのinstallでは 不要な操作だったようです。

libgeos_c.so.1: cannot open shared object file: No such file or directory

$ /usr/local/pgsql/bin/psql -U postgres -d gis_test \
   -f /usr/local/pgsql/share/contrib/postgis-2.5/postgis.sql
BEGIN
SET
DO
CREATE FUNCTION
psql:/usr/local/pgsql/share/contrib/postgis-2.5/postgis.sql:94: ERROR:  could not load library "/usr/local/pgsql/lib/postgis-2.5.so": libgeos_c.so.1: cannot open shared object file: No such file or directory
   :

のようなエラーが発生。

$ sudo vi /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/local/lib

$ sudo ldconfig

のように「/usr/local/lib」を追加し、一旦?は回避しました。

mapserverno cmakeコマンド

以下のように実行しました。 この後の 国土交通省にあるGISデータをPostGISへインポートやPNGファイル作成も問題なく動作しています。

$ cmake -DCMAKE_PREFIX_PATH=/usr/local/pgsql \
>       -DWITH_PROTOBUFC=0 \
>       -DWITH_GIF=0 \
>       ..
-- * Summary of configured options for this build
--  * Mandatory components
--   * png: /usr/local/lib/libpng.so
--   * jpeg: /usr/local/lib/libjpeg.so
--   * freetype: /usr/local/lib/libfreetype.so
--  * Optional components
--   * GDAL: /usr/local/lib/libgdal.so
--   * OGR: /usr/local/lib/libgdal.so
--   * GIF: disabled
--   * MYSQL: disabled
--   * FRIBIDI: /usr/local/lib/libfribidi.so
--   * HARFBUZZ: /usr/lib64/libharfbuzz.so
--   * GIF: disabled
--   * CAIRO: /usr/lib64/libcairo.so
--   * SVGCAIRO: disabled
--   * RSVG: disabled
--   * CURL: disabled
--   * PROJ: /usr/local/lib/libproj.so
--   * PIXMAN: disabled
--   * LIBXML2: /usr/lib64/libxml2.so
--   * POSTGIS: /usr/local/pgsql/lib/libpq.so
--   * GEOS: /usr/local/lib/libgeos_c.so
--   * FastCGI: /usr/local/lib/libfcgi.so
--   * PROTOBUFC: disabled
--   * Oracle Spatial: disabled
--   * Exempi XMP: disabled
--  * Optional features
--   * WMS SERVER: ENABLED
--   * WFS SERVER: ENABLED
--   * WCS SERVER: ENABLED
--   * SOS SERVER: disabled
--   * WMS CLIENT: disabled
--   * WFS CLIENT: disabled
--   * ICONV: ENABLED
--   * Thread-safety support: disabled
--   * KML output: disabled
--   * Z+M point coordinate support: disabled
--   * XML Mapfile support: disabled
--  * Mapscripts
--   * Python: disabled
--   * PHP: disabled
--   * PERL: disabled
--   * RUBY: disabled
--   * JAVA: disabled
--   * C#: disabled
--   * V8 Javascript: disabled
--   * Apache Module (Experimental): disabled
-- 
-- Will install files to /usr/local
-- Will install libraries to /usr/local/lib
-- Configuring done
-- Generating done
-- Build files have been written to: /home/end0tknr/tmp/mapserver-7.2.2/build

centos 7 ( systemd )で postgresql-11 を自動起動

$ tar -xvf postgresql-11.2.tar.gz
$ cd postgresql-11.2
$ less INSTALL
        --with-systemd
                Build with support for systemd service notifications. This
                improves integration if the server binary is started under
                systemd but has no impact otherwise. libsystemd and the
                associated header files need to be installed to be able to
                use this option.

と記載されいたので、「--with-systemd」 付きでconfigureしたら

$ ./configure --with-systemd
  :
checking systemd/sd-daemon.h usability... no
checking systemd/sd-daemon.h presence... no
checking for systemd/sd-daemon.h... no
configure: error: header file <systemd/sd-daemon.h> is required for systemd support

と、エラー

$ sudo yum install systemd-devel 

のように systemd-devel が必要らしい。

後は、 https://www.postgresql.org/docs/11/server-start.html に記載の通り postgresql.service を作成し、登録すればOK

$ sudo vi /etc/systemd/system/postgresql.service

[Unit]
Description=PostgreSQL database server
Documentation=man:postgres(1)

[Service]
Type=notify
User=postgres
ExecStart=/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
ExecReload=/bin/kill -HUP $MAINPID
KillMode=mixed
KillSignal=SIGINT
TimeoutSec=0

[Install]
WantedBy=multi-user.target

その他、make 手順はINSTALLファイルにコマンドラインレベルで記載されています

python v2.7 + selenium + firefox (geckodriver.exe) でfileの自動upload & download

多分、↓こう

#!/usr/local/bin/python
# -*- coding: utf-8 -*-
import getopt
import os
import sys
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import traceback
from time import sleep

CONF = {'mile_id':'ないしょ','mile_pw':'ないしょ'}

def main():
    # browser起動
    browser = init_browser()
    # login処理
    if login_group_ware(browser) == False:
        return False
    # file upload
    file_name = "browser_screen_shot.png"
    if upload_file(browser,file_name) == False:
        return False
    # file download
    download_file(browser,file_name)
    # 終了処理
    browser.quit()

    
def download_file(browser,file_name):

    print file_name
    
    browser.get("https://bunsho.mile.sexy.com/lite/folder/204939")

    tr_lists = browser.find_elements_by_class_name("listItemName")
    for tr_list in tr_lists:
        if tr_list.text == file_name:
            a_href = tr_list.find_element_by_tag_name("a")
            a_href.click()
            sleep(5)

            download_link = browser.find_element_by_id("BunshoVS_dlLink")
            download_link.click()
            browser.implicitly_wait(10) # seconds
            return True
    return False

        
def upload_file(browser, file_name):

    browser.get("https://bunsho.mile.sexy.com/lite/folder/204939")


    browser.get("https://bunsho.mile.sexy.com/folder/204939/file/create")
    browser.execute_script('javascript:switch_to_normal_mode()')

    elm_file = browser.find_element_by_id("fp0file")
    elm_file.send_keys("c:\\home\\end0tknr\\tmp\\SELENIUM\\" + file_name)
    
    browser.execute_script('javascript:Creator.create_commit(file_select_mode)')
    
    # seleniumにおけるwaitは以下を参照
    # refer to https://kurozumi.github.io/selenium-python/waits.html
    try:
        WebDriverWait(browser, 10).until(
            EC.title_contains(u"ほげほげ")
        )
    except Exception as e:
#        print(dir(e))
        print e.args
        return False

    return True

    
def init_browser():
    profile = webdriver.FirefoxProfile()

    # 0:desktop, 1:sys規定のfolder, 2:user def folder
    profile.set_preference("browser.download.folderList",2)
    profile.set_preference("browser.download.dir", os.getcwd())
    profile.set_preference("browser.download.manager.showWhenStarting",False)

    # chrome driverでは、download.directory_upgrade=Trueで上書き保存できましたが
    # firefox では、上書き方法不明
    # profile.set_preference("browser.download.directory_upgrade",True)

    # image/png の場合、強制的にdownload (=画面表示しない)
    profile.set_preference("browser.helperApps.neverAsk.saveToDisk","image/png")
    browser = webdriver.Firefox(firefox_profile=profile)

    return browser

    
def login_group_ware(browser):

    browser.get("https://login.mile.sexy.com/login.pl")
    login_id_form = browser.find_element_by_name('credential_0')
    login_id_form.send_keys(CONF['mile_id'])
    login_pw_form = browser.find_element_by_name('credential_1')
    login_pw_form.send_keys(CONF['mile_pw'])

    login_btn = browser.find_element_by_name('Submit')
    login_btn.click()

    if( browser.current_url == "https://top.mile.sexy.com/"):
        return True
    return False


if __name__ == '__main__':
    main()

linux netstat or lsof でポートを使用するプロセス(プログラム)を確認

sonarqube-7.7 を install後、起動を試みるも、できなかったので、調査。

$ ./sonar.sh console
Running SonarQube...
wrapper  | --> Wrapper Started as Console
wrapper  | Launching a JVM...
jvm 1    | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
jvm 1    |   Copyright 1999-2006 Tanuki Software, Inc.  All Rights Reserved.
jvm 1    | 
jvm 1    | 2019.04.02 20:24:47 INFO  app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /home/end0tknr/local/sonarqube-7.7/temp
jvm 1    | 2019.04.02 20:24:47 INFO  app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
jvm 1    | 2019.04.02 20:24:48 INFO  app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/home/end0tknr/local/sonarqube-7.7/elasticsearch]: /home/end0tknr/local/sonarqube-7.7/elasticsearch/bin/elasticsearch
jvm 1    | 2019.04.02 20:24:48 INFO  app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm 1    | 2019.04.02 20:24:49 INFO  app[][o.e.p.PluginsService] no modules loaded
jvm 1    | 2019.04.02 20:24:49 INFO  app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm 1    | OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
jvm 1    | 2019.04.02 20:25:38 WARN  app[][o.s.a.p.AbstractProcessMonitor] Process exited with exit value [es]: 1
jvm 1    | 2019.04.02 20:25:38 INFO  app[][o.s.a.SchedulerImpl] Process [es] is stopped
jvm 1    | 2019.04.02 20:25:38 INFO  app[][o.s.a.SchedulerImpl] SonarQube is stopped
wrapper  | <-- Wrapper Stopped

で、以下の通り

$ sudo netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:5000            0.0.0.0:*               LISTEN      3570/perl           
tcp        0      0 127.0.0.1:9001          0.0.0.0:*               LISTEN      3037/python         
tcp        0      0 0.0.0.0:139             0.0.0.0:*               LISTEN      3117/smbd           
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      3041/sshd           
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      3568/master         
tcp        0      0 0.0.0.0:445             0.0.0.0:*               LISTEN      3117/smbd           
tcp6       0      0 :::3306                 :::*                    LISTEN      3328/mysqld         
tcp6       0      0 :::139                  :::*                    LISTEN      3117/smbd           
tcp6       0      0 :::80                   :::*                    LISTEN      3035/httpd          
tcp6       0      0 :::22                   :::*                    LISTEN      3041/sshd           
tcp6       0      0 ::1:25                  :::*                    LISTEN      3568/master         
tcp6       0      0 :::445                  :::*                    LISTEN      3117/smbd           
$

更に lsof コマンドで詳細確認

$ sudo lsof -i :9001
COMMAND    PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
superviso 3037 root    4u  IPv4  24613      0t0  TCP localhost:etlservicemgr (LISTEN)
$

最終的に、supervisord が9001ポートを既に使用していることが分かります

lizard (python製) で、C# (csharp) の 循環的複雑度 (cyclomatic_complexity)計測

https://github.com/terryyin/lizard

c# の code metricsは、visual studio で計測できますが、 全srcの循環的複雑度をコマンドラインから一覧で生成したかった為、lizard を利用。

install & usage

$ sudo /usr/local/bin/pip install lizard

$ /usr/local/bin/lizard --version
1.16.3
$ /usr/local/bin/lizard --help
usage: lizard [options] [PATH or FILE] [PATH] ...

lizard is an extensible Cyclomatic Complexity Analyzer for many programming
languages including C/C++ (doesn't require all the header files). For more
information visit http://www.lizard.ws

positional arguments:
  paths                 list of the filename/paths.

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -l LANGUAGES, --languages LANGUAGES
                        List the programming languages you want to analyze. if
                        left empty, it'll search for all languages it knows.
                        `lizard -l cpp -l java`searches for C++ and Java code.
                        The available languages are: cpp, java, csharp,
                        javascript, python, objectivec, ttcn, ruby, php,
                        swift, scala, GDScript, go, lua
  -V, --verbose         Output in verbose mode (long function name)
  -C CCN, --CCN CCN     Threshold for cyclomatic complexity number warning.
                        The default value is 15. Functions with CCN bigger
                        than it will generate warning
  -f INPUT_FILE, --input_file INPUT_FILE
                        get a list of filenames from the given file
  -L LENGTH, --length LENGTH
                        Threshold for maximum function length warning. The
                        default value is 1000. Functions length bigger than it
                        will generate warning
  -a ARGUMENTS, --arguments ARGUMENTS
                        Limit for number of parameters
  -w, --warnings_only   Show warnings only, using clang/gcc's warning format
                        for printing warnings.
                        http://clang.llvm.org/docs/UsersManual.html#cmdoption-
                        fdiagnostics-format
  --warning-msvs        Show warnings only, using Visual Studio's warning
                        format for printing warnings.
                        https://msdn.microsoft.com/en-us/library/yxkt8b26.aspx
  -i NUMBER, --ignore_warnings NUMBER
                        If the number of warnings is equal or less than the
                        number, the tool will exit normally; otherwise, it
                        will generate error. If the number is negative, the
                        tool exits normally regardless of the number of
                        warnings. Useful in makefile for legacy code.
  -x EXCLUDE, --exclude EXCLUDE
                        Exclude files that match the pattern. * matches
                        everything, ? matches any single character,
                        "./folder/*" exclude everything in the folder
                        recursively. Multiple patterns can be specified. Don't
                        forget to add "" around the pattern.
  -t WORKING_THREADS, --working_threads WORKING_THREADS
                        number of working threads. The default value is 1.
                        Using a bigger number can fully utilize the CPU and
                        often faster.
  -X, --xml             Generate XML in cppncss style instead of the tabular
                        output. Useful to generate report in Jenkins server
  --csv                 Generate CSV output as a transform of the default
                        output
  -H, --html            Output HTML report
  -m, --modified        Calculate modified cyclomatic complexity number ,
                        which count a switch/case with multiple cases as one
                        CCN.
  -E EXTENSIONS, --extension EXTENSIONS
                        User the extensions. The available extensions are:
                        -Ecpre: it will ignore code in the #else branch.
                        -Ewordcount: count word frequencies and generate tag
                        cloud. -Eoutside: include the global code as one
                        function. -EIgnoreAssert: to ignore all code in
                        assert. -ENS: count nested control structures.
  -s SORTING, --sort SORTING
                        Sort the warning with field. The field can be nloc,
                        cyclomatic_complexity, token_count, p#arameter_count,
                        etc. Or an customized field.
  -T THRESHOLDS, --Threshold THRESHOLDS
                        Set the limit for a field. The field can be nloc,
                        cyclomatic_complexity, token_count, parameter_count,
                        etc. Or an customized file. Lizard will report warning
                        if a function exceed the limit
  -W WHITELIST, --whitelist WHITELIST
                        The path and file name to the whitelist file. It's
                        './whitelizard.txt' by default. Find more information
                        in README.
$

sample

$ /usr/local/bin/lizard --exclude "./GRA_batch_report/Backup/*" \
                                  --sort cyclomatic_complexity \
                  --languages csharp . > foo.txt
$ cat foo.txt
================================================
  NLOC    CCN   token  PARAM  length  location
------------------------------------------------
       4      1      9      0       4 CM.rpt::CM_A?_0?_R?2::CM_A?_0?_R02@34-37@./GRA_batch_report/ba_??_a??_0??_r02/rpt/CM_A??_0??_R0??.cs
      11      1     91      2      13 CM.rpt::CM_A?_0?_R?2::CM_A?_0?_R02_Re???Start@63-75@./GRA_batch_report/ba_cm_??_03_r02/rpt/CM_??_03_R02.cs
    <略>
      37      3    227      2      44 CM::BL_CM_M?_1?_???4::Get??ti??Grp@430-473@./GRA_web/bl_??_ms_11/??_CM_??_11_S04.cs
1074 file analyzed.

==============================================================
NLOC    Avg.NLOC  AvgCCN  Avg.token  function_cnt    file
--------------------------------------------------------------
   1475     228.2     1.7     2709.0         6     ./GRA_batch_report/ba_??_??_03_r02/rpt/??_AJ_??_R02.cs
     14       0.0     0.0        0.0         0     ./GRA_batch_report/ba_??_??_03_r02/Asse??yInfo.cs
    <略>
      0       0.0     0.0        0.0         0     ./GRA_web/bl_??_??_11/BL_??_??_11_S05.cs

=========================================================================================
!!!! Warnings (cyclomatic_complexity > 15 or length > 1000 or parameter_count > 100) !!!!

================================================
  NLOC    CCN   token  PARAM  length  location  
------------------------------------------------
    1369    313   8701      2    1807 FI::FI_??_03_??2::Da????id1_Item??ta??und@2503-4309@./GRA_web/allegroweb/FI_GL_03_S02.aspx.cs
     981    222   6540      2    1252 FI::FI_??_03_??9::Da????id1_Item??ta??und@950-2201@./GRA_web/allegroweb/FI_GL_03_S09.aspx.cs
    <略>
    1313      1  16162      0    1521 FI.rpt::??_??_05_R12::Initi??zeCo??nent@307-1827@./GRA_web/allegroweb/rpt/XF_GL_05_R12.cs
==========================================================================================
Total nloc   Avg.NLOC  AvgCCN  Avg.token   Fun Cnt  Warning cnt   Fun Rt   nloc Rt
------------------------------------------------------------------------------------------
    871324      49.8     4.5      446.4    15996          885      0.06    0.56

linux の cat やgrepは、PowerShellでは、Get-Content や Select-String

cat ≒ Get-Content

以下の通り、defaultでは、sjis以外文字化けするようです

PS C:\home\end0tknr\tmp\PS1> Get-Content .\foo_sjis.txt
これは、SJISのファイルです
PS C:\home\end0tknr\tmp\PS1> Get-Content -Encoding UTF8 .\foo_utf8.txt
これは、UTF-8のファイルです

grep ≒ Select-String

以下の通り、defaultでは、大文字小文字を区別しないようです

PS C:\home\end0tknr\tmp\PS1> Get-Content -Encoding UTF8 .\foo_utf8.txt | Select-String -Pattern "UTF" -CaseSensitive

これは、UTF-8のファイルです

(再) サーバにinstallされているOSやMWの一覧を自動収集したい (windows版)

サーバにinstallされているOSやMWの一覧を自動収集したい (windows版) - end0tknr's kipple - 新web写経開発

先日、記載したエントリではワンライナーで、読みづらいので power shell script らしく?修正しました。(以下)

function do_main {

    $i = 0

    #### win os 情報の収集
    $os_info = collect_win_os_ver
    $disp_cols = @([string] $i++,
           $os_info.SystemDirectory,
           $os_info.Version,
           $os_info.RegisteredUser )

    # タブ区切りで出力
    $disp_str = [string]::Join("`t", $disp_cols)
    # Write-Host と Write-Output の違いは要注意
    Write-Output $disp_str

    
    #### strawberry perl のver
    $disp_str = collect_perl_ver
    $disp_cols = @([string] $i++,
           "perl"
           $disp_str)
    $disp_str = [string]::Join("`t", $disp_cols)
    Write-Output $disp_str

    #### java spring のver
    $disp_str = collect_java_spring_ver
    $disp_cols = @([string] $i++,
           "java spring"
           $disp_str)
    $disp_str = [string]::Join("`t", $disp_cols)
    Write-Output $disp_str

    #### installerでinstallされたアプリ情報の収集
    $pkg_infos = collect_normal_package
    ## [string] $i は、数値の文字列化
    $i = 0
    foreach ($pkg_info in $pkg_infos) {

    if($pkg_info.DisplayName.Length -eq 0 -and
       $pkg_info.DisplayVersion.Length -eq 0 -and
       $pkg_info.Publisher.Length -eq 0 ){
           continue
       }

    $disp_cols = @([string] $i++,
               $pkg_info.DisplayName,
               $pkg_info.DisplayVersion,
               $pkg_info.Publisher)
    $disp_str = [string]::Join("`t", $disp_cols)
    Write-Output $disp_str
    }


}


function collect_java_spring_ver {

    $gradle_file =
    "c:\home\end0tknr\eclipse-workspace-oxygen\SpringBootMyBatis\build.gradle"
    $item_prop =
    Get-Content -Encoding UTF8 $gradle_file | Select-String "springBootVersion ="

    return $item_prop.toString().trim()
}


function collect_perl_ver {
    $item_prop = perl -v | Select-String -Pattern "version"
    return $item_prop.toString().trim()
}

function collect_win_os_ver {
    $item_prop =
    @(Get-WmiObject Win32_OperatingSystem |
      Select-Object SystemDirectory, Version, RegisteredUser )

    return $item_prop
}


function collect_normal_package {
    $pkg_infos =
    @( Get-ChildItem -Path(
           'HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall',
           'HKCU:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall'))
    
    $ret_pkg_infos = @()
    foreach ($pkg_info in $pkg_infos) {
    $item_prop =
    @(Get-ItemProperty $pkg_info.PsPath |
      Select-Object DisplayName, DisplayVersion, Publisher )

    $ret_pkg_infos += $item_prop
    }
    return $ret_pkg_infos
}

# function collect_normal_package {
#     $pkg_infos =
#     @( Get-ChildItem -Path(
#      'HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall',
#      'HKCU:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall') |
#        % { Get-ItemProperty $_.PsPath |
#        Select-Object DisplayName, DisplayVersion, Publisher } )
#     return $pkg_infos
# }


do_main

(再) install WindowsApplicationDriver

https://end0tknr.hateblo.jp/entry/20190323/1553338368

先日、上記entryにて、Windows Application Driver をinstallしていますが、 あまりに中途半端なので、再度、hands-on。

find_element(s)by_accessibility_id() , find_element(s)by_class_name() , find_element(s)by_id() , find_element(s)by_name() , find_element(s)by_tag_name() , find_element(s)by_xpath() 等、要素探索のmethodが用意されていますが、 「WinAppDriver UI Recorder」でXPATHを調べ find_element_by_xpath()による要素探索が最も良かった。

ただし、アプリによっては「WinAppDriver UI Recorder」や「inspect.exe」による 調査中にアプリが強制終了する点が、なかなか難しい。

事前準備

Windows Application Driver

以下の WinAppDriver.exe を起動すると、port:4723 経由でwindowsアプリを操作できます。

WinAppDriver UI Recorder

https://github.com/Microsoft/WinAppDriver/releases/download/UiR_v1.0-RC/WinAppDriverUIRecorder.zip

上記zipを解凍し、 WinAppDriverUiRecorder.exe を起動すると、 アプリの各要素のXPATHを調べることができます。

inspect.exe

「WinAppDriver UI Recorder」と同様ですが、 アプリの各要素のクラス名や要素名等の様々な項目を調べることができます。

https://developer.microsoft.com/ja-jp/windows/downloads/sdk-archive からダウンロードし、インストールすると、以下に配備されます。

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\inspect.exe

https://developer.microsoft.com/ja-jp/windows/downloads/sdk-archive

sample script

# -*- coding: utf-8 -*-
from appium import webdriver
import time

def main():
    ## アプリ起動
    desired_caps = {}
    desired_caps["app"] = r"C:\SCOOP\SCOOP22\Program\PLANMAN.exe"
    driver = webdriver.Remote(
        command_executor='http://127.0.0.1:4723',
        desired_capabilities= desired_caps)

    win_planman = init_planman(driver)
    print win_planman

    ## ↑ここまではうまく動作しました。
    ## ↓この先で、ListViewやTreeViewを探索したかったのですが
    ##   アプリが強制終了するなど、うまく動作しませんでした。
    pane_tree = win_planman.find_elements_by_class_name("ListLiew")
    print pane_tree

    
    ## 終了
    driver.quit()

def init_planman(driver):
    ## logon画面から遷移
    xpath_str = \
        "/".join(['/Pane[@Name="デスクトップ 1"][@ClassName="#32769"]',
                  'Window[@Name="ログオン"][@ClassName="ThunderRT6FormDC"]'])
    diagram_logon = driver.find_element_by_xpath(xpath_str)

    xpath_str = \
        "//Button[@Name=\"OK\"][@ClassName=\"ThunderRT6CommandButton\"]"
    diagram_logon.find_element_by_xpath(xpath_str).click()

    
    ## version check画面から遷移
    xpath_str = \
        "/".join(['/Pane[@Name="デスクトップ 1"][@ClassName="#32769"]',
                  'Window[@Name="SCOOP22 マネージャ"][@ClassName="#32770"]'])
    try:
        diagram_ver_chk = driver.find_element_by_xpath(xpath_str)
    except Exception as e:
        pass  ## version check画面が単に見つからない場合、無視して進みます
    else:
        xpath_str = "//Button[@Name=\"OK\"][@ClassName=\"Button\"]"
        diagram_ver_chk.find_element_by_xpath(xpath_str).click()
    
    ## plan manager探索
    xpath_str = \
        "/".join(['/Pane[@Name="デスクトップ 1"][@ClassName="#32769"]',
                  'Window[@Name="SCOOP22 マネージャ - soop22"]'+
                  '[@ClassName="ThunderRT6FormDC"]'])
    try:
        win_planman = driver.find_element_by_xpath(xpath_str)
    except Exception as e:
        print e , " " , xpath_str
        return

    return win_planman


if __name__ == '__main__':
    main()

サーバにinstallされているOSやMWの一覧を自動収集したい (windows版)

サーバにinstallされているOSやMWの一覧を自動収集したい (centos版) - end0tknr's kipple - 新web写経開発

先日の上記エントリのwindows版。

windowsの場合、コマンドライン(power shell Get-ChildItem)から以下のようにインストール済の一覧を取得できます

PS C:\Users\end0t> Get-ChildItem -Path(
>> 'HKLM:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall',
>> 'HKCU:SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall') |
>> % { Get-ItemProperty $_.PsPath | Select-Object DisplayName, DisplayVersion, Publisher }

DisplayName                                                           DisplayVersion  Publisher
-----------                                                           --------------  ---------
7-Zip 18.05 (x64)                                                     18.05           Igor Pavlov
Android Studio                                                        3.3             Google LLC
EPSON PX-049A Series プリンター アンインストール                                      Seiko Epson Corporation
Appium 1.11.0                                                         1.11.0          Appium Developers
GIMP 2.10.6                                                           2.10.6          The GIMP Team
Git version 2.19.1                                                    2.19.1          The Git Development Community
Mozilla Firefox 64.0.2 (x64 ja)                                       64.0.2          Mozilla
Mozilla Maintenance Service                                           61.0            Mozilla
Microsoft Project Professional 2016                                   16.0.4266.1001  Microsoft Corporation
Microsoft Office Professional Plus 2016                               16.0.4266.1001  Microsoft Corporation
Microsoft Visio Professional 2016                                     16.0.4266.1001  Microsoft Corporation
OWASP Zed Attack Proxy 2.7.0                                          2.7.0           OWASP ZAP
Vulkan Run Time Libraries 1.1.70.0                                    1.1.70.0        LunarG, Inc.
Canon MP470 series
Windows SDK for Windows Store Apps DirectX x64 Remote                 10.1.16299.91   Microsoft Corporation
Intel(R) Management Engine Components                                 11.7.0.1035     Intel Corporation
Application Verifier x64 External Package                             10.1.16299.91   Microsoft
Intel(R) PRO/Wireless Driver                                          20.50.0001.8400 Intel Corporation
Intel(R) Management Engine Components                                 1.0.0.0         Intel Corporation
Microsoft Visual C++ 2017 X64 Debug Runtime - 14.16.27024             14.16.27024     Microsoft Corporation
Windows App Certification Kit Native Components                       10.1.17763.132  Microsoft Corporation
Java(TM) SE Development Kit 10.0.1 (64-bit)                           10.0.1.0        Oracle Corporation
Universal CRT Tools x64                                               10.1.17763.132  Microsoft Corporation
Windows SDK for Windows Store Apps DirectX x64 Remote                 10.1.17134.12   Microsoft Corporation
Lenovo Active Protection System                                       1.82.00.17      Lenovo
CMake                                                                 3.14.0          Kitware
TortoiseSVN 1.10.1.28295 (64 bit)                                     1.10.28295      TortoiseSVN
   :

install WindowsApplicationDriver

またも「RPA風に使えるかも」関連.

appium & Windows Application Driver 経由でのwindowsアプリ操作を期待しましたが、 まず、appium の起動は不要でした。 また、単純なボタンのクリック等はできましたが、要素の探索等はできませんでした。 (詳しいドキュメントを見つけることができなかった為)

参考url

install WindowsApplicationDriver 関連

WindowsApplicationDriver自体は、 https://github.com/Microsoft/WinAppDriver から インストーラをダウンロードし、実行するだけ。

更にnodejsやpythonのモジュールをインストールすれば完了

C:\Users\end0t>npm install --global --production windows-build-tools
C:\Users\end0t>npm install appium-windows-driver
C:\Users\end0t>pip install Appium-Python-Client
C:\Users\end0t>pip install setuptools

install inspect.exe

捜査対象のアプリの要素名を調べる為、inspect.exe もあると便利です

https://developer.microsoft.com/ja-jp/windows/downloads/sdk-archive からダウンロードし、インストールすると、以下に配備されます。

C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64\inspect.exe

軽く動作確認

PS C:\Program Files (x86)\Windows Application Driver> pwd

Path
----
C:\Program Files (x86)\Windows Application Driver


PS C:\Program Files (x86)\Windows Application Driver> .\WinAppDriver.exe
Windows Application Driver listening for requests at: http://127.0.0.1:4723/
Press ENTER to exit.

↑こちらのように WinAppDriver.exe を起動した状態で ↓microsoftから配布されているサンプルscriptが動作すればOK

"""
//******************************************************************************
//
// Copyright (c) 2016 Microsoft Corporation. All rights reserved.
//
// This code is licensed under the MIT License (MIT).
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
// THE SOFTWARE.
//
//******************************************************************************
"""

import unittest
from appium import webdriver

class SimpleCalculatorTests(unittest.TestCase):

    def setUp(self):
        #set up appium
        desired_caps = {}
        desired_caps["app"] = "Microsoft.WindowsCalculator_8wekyb3d8bbwe!App"
        self.driver = webdriver.Remote(
            command_executor='http://127.0.0.1:4723',
            desired_capabilities= desired_caps)

    def tearDown(self):
        self.driver.quit()

    def test_initialize(self):
        self.driver.find_element_by_accessibility_id("clearButton").click()
        self.driver.find_element_by_accessibility_id("num7Button").click()

        result = self.driver.find_element_by_accessibility_id("CalculatorResults")
        self.assertIn(" 7 ", str(result.text))

    def test_addition(self):
        self.driver.find_element_by_accessibility_id("num1Button").click()
        self.driver.find_element_by_accessibility_id("plusButton").click()
        self.driver.find_element_by_accessibility_id("num7Button").click()
        self.driver.find_element_by_accessibility_id("equalButton").click()

        result = self.driver.find_element_by_accessibility_id("CalculatorResults")
        self.assertIn( " 8 ", str(result.text))

    def test_combination(self):
        self.driver.find_element_by_accessibility_id("num7Button").click()
        self.driver.find_element_by_accessibility_id("multiplyButton").click()
        self.driver.find_element_by_accessibility_id("num9Button").click()
        self.driver.find_element_by_accessibility_id("plusButton").click()
        self.driver.find_element_by_accessibility_id("num1Button").click()
        self.driver.find_element_by_accessibility_id("equalButton").click()
        self.driver.find_element_by_accessibility_id("divideButton").click()
        self.driver.find_element_by_accessibility_id("num8Button").click()
        self.driver.find_element_by_accessibility_id("equalButton").click()

        result = self.driver.find_element_by_accessibility_id("CalculatorResults")
        self.assertIn( " 8 ", str(result.text))

    def test_division(self):
        self.driver.find_element_by_accessibility_id("num8Button").click()
        self.driver.find_element_by_accessibility_id("num8Button").click()
        self.driver.find_element_by_accessibility_id("divideButton").click()
        self.driver.find_element_by_accessibility_id("num1Button").click()
        self.driver.find_element_by_accessibility_id("num1Button").click()
        self.driver.find_element_by_accessibility_id("equalButton").click()

        result = self.driver.find_element_by_accessibility_id("CalculatorResults")
        self.assertIn( " 8 ", str(result.text))

    def test_multiplication(self):
        self.driver.find_element_by_accessibility_id("num9Button").click()
        self.driver.find_element_by_accessibility_id("multiplyButton").click()
        self.driver.find_element_by_accessibility_id("num9Button").click()
        self.driver.find_element_by_accessibility_id("equalButton").click()

        result = self.driver.find_element_by_accessibility_id("CalculatorResults")
        self.assertIn( " 81 ", str(result.text))

    def test_subtraction(self):
        self.driver.find_element_by_accessibility_id("num9Button").click()
        self.driver.find_element_by_accessibility_id("minusButton").click()
        self.driver.find_element_by_accessibility_id("num1Button").click()
        self.driver.find_element_by_accessibility_id("equalButton").click()

        result = self.driver.find_element_by_accessibility_id("CalculatorResults")
        self.assertIn( " 8 ", str(result.text))



if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(SimpleCalculatorTests)
    unittest.TextTestRunner(verbosity=2).run(suite)

開発者モード設定画面で、エラー?が表示されましたが、気にせず進めてもよさそうです

f:id:end0tknr:20190323195602p:plain

install appium and Appium-Python-Client to windows 10

「"RPA"風に利用できるかも」と思い、次のurlを参考にさせて頂きながら、 appium を windows10にinstall 。

今回は「とりあえずのinstall」まで行っています。

java , python , nodejs

それぞれが、何の prerequired かであるかは、忘れましたが、 appium 等のinstall 前に以下のversionがinstallされていました。

C:\Users\end0t>java -version
java version "10.0.1" 2018-04-17

C:\Users\end0t>python --version
Python 2.7.16

C:\Users\end0t>node -v
  v10.15.3
C:\Users\end0t>npm -v
  6.4.1

install android studio (3.3.2 for Windows 64-bit (948 MB))

android仮想端末とplatform-toolsが必要ですので、 android studio をinstallしています。

インストーラandroid-studio-ide-182.5314842-windows.exe です。 (ver.3.3.2 という文字列がファイル名にないことが少々、気になります)

android studioのinstallにより、SDKも合わせてinstallされますが windows環境変数に以下を追加しています。

android studioから仮想端末を作成し、adbで動作確認

次のように追加した仮想端末が認識されていれば、OK

C:\Users\end0t>adb devices
List of devices attached
emulator-5554   device

install appium (GUI版)

http://appium.io/ から appium-desktop-setup-1.11.0.exe をダウンロードし、installするだけです。

install appium (CUI版)

C:\Users\end0t>npm install -g appium
C:\Users\end0t>npm install -g appium-doctor
C:\Users\end0t>appium-doctor

install後の appium の状態?は、appium-doctor が診断してくれます。 私の場合、 windows-build-tools や opencv4nodejs をinstallしています。 ( cmake もinstallしたような)

PS C:\WINDOWS\system32> npm install --global windows-build-tools
PS C:\WINDOWS\system32> npm install --save opencv4nodejs

install Appium-Python-Client

C:\Users\end0t>pip install Appium-Python-Client

軽く動作確認

#!python
# -*- coding: utf-8 -*-
import getopt
import sys
import os
from appium import webdriver
from appium.webdriver.common.touch_action import TouchAction
import unittest


# https://github.com/appium/python-client

class TestAppium(unittest.TestCase):

    def setUp(self):
        desired_caps = {}
        desired_caps["platformName"] = "Android"
        desired_caps["automationName"] = "Appium"
#        desired_caps["automationName"] = "uiautomator2"
        desired_caps['app'] = os.path.abspath(os.path.join(os.path.dirname(__file__),'ApiDemos-debug.apk'))
        desired_caps["platformVersion"] =  "8.1"
        desired_caps['deviceName'] = 'Nexus 5 API 27'
        desired_caps["appPackage"] = "io.appium.android.apis"
        desired_caps["appActivity"] = "io.appium.android.apis.ApiDemos"
        
        self.driver = webdriver.Remote('http://localhost:4723/wd/hub',
                                       desired_caps)

    def tearDown(self):
        self.driver.quit()

    def test_contexts_list(self):
        el = self.driver.find_element_by_class_name('android.widget.ListView')

        actions = TouchAction(self.driver)
        actions.flick_element(el, 0, -300, 0)
        actions.perform()
        sleep(5)


#    def test_find_element(self):

#        elm = self.driver.find_element_by_class_name('android.widget.TextView');
#        print(elm)
#        elm.click()
        
#        TouchAction(self.driver).tap(x=204, y=530).perform()
#        TouchAction(self.driver).tap(x=204, y=700).perform()

        # 要素が見つかるまで ?sec待つ
#        self.driver.implicitly_wait(5)
        
#        elm = self.driver.find_element_by_accessibility_id("App");
#        elm = self.driver.find_element_by_id("android:id/text7");


if __name__ == "__main__":
    unittest.main()

android studio v3.3.2 for Windows 64-bit のinstall時に platform-toolsがinstallされない

以前、installした android studio v3.1 をuninstallし、 android studio v3.3 をre-installしたことが原因でしょうか?

  • C:\Users\end0t.AndroidStudio3.1
  • C:\Users\end0t.AndroidStudio3.3

android studio をuninstallしても削除されない、上記profileを削除後、 再installすることで解消しました。

f:id:end0tknr:20190322081048p:plain

install後は、↓この辺りの環境変数設定も忘れないように - ANDROID_HOME = C:\Users\end0t\AppData\Local\Android\Sdk - PATH = %ANDROID_HOME%\platform-tools