end0tknr's kipple - web写経開発

太宰府天満宮の狛犬って、妙にカワイイ

shell scriptとバイナリファイルを1つのファイルにまとめる

https://github.com/Microsoft/OMS-Agent-for-Linux/releases

にある omsagent-?.?.?-256.universal.x86.sh は、shell scriptなのに ファイルサイズが大きいので、中身を読んでみたら shell script にバイナリファイル(*.tar)をまとめていた。

多分、バイナリファイルを cat し、 shell script へマージしたのかと思います。

使用する際は、自信のscriptを tail していましたが、このアイデアは、私にはなかった...

aws s3にあるファイルをバージョンID付きで削除 - perl

aws s3には、バージョニングという機能があり、 単純にファイル削除した場合、実体は削除されず、 削除フラグ=ON(つまり、論理削除)の状態になるらしい。

物理削除したい場合、バージョンID付きで削除すればよいらしく、perlで書くと、以下の通り。

  • Net::Amazon::S3 * を使用したかったのですが、 Net::Amazon::S3 によるバージョンID付き削除方法が分からなかった為、 次のように素朴なperl scriptにしました。
#!/usr/local/perl/bin/perl
use strict;
use warnings;
use utf8;
use Digest::HMAC_SHA1;
use File::Basename;
use File::stat;
use HTTP::Date;
use HTTP::Request;
use HTTP::Request::Common;
use LWP::UserAgent;
use Log::Log4perl;
use Data::Dumper;

# https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/RESTAuthentication.html
# http://blog.yusuke.be/entry/2014/01/23/011321
my $CONF =
    {aws_s3=>
     {host=>                   'ないしょ',
      aws_access_key_id =>     'ないしょ',
      aws_secret_access_key => 'ないしょ',
      bucket=>                 'ないしょ'},
     chunk_size_limit => 1000,
     chunk_interval   => 10,  #sec
     log=>
     {'log4perl.rootLogger'=> 'DEBUG, LOGFILE, CONSOLE',
      'log4perl.appender.LOGFILE'=>'Log::Log4perl::Appender::File',
      'log4perl.appender.LOGFILE.dir'=>'/home/end0tknr/delete/',
      'log4perl.appender.LOGFILE.filename'=>
      '/home/end0tknr/delete/delete.log',
      'log4perl.appender.LOGFILE.mode'=>'append',
      'log4perl.appender.LOGFILE.layout'=>'Log::Log4perl::Layout::PatternLayout',
      'log4perl.appender.LOGFILE.layout.ConversionPattern'=>'%d [%p] %m %n',

      'log4perl.appender.CONSOLE'=> 'Log::Log4perl::Appender::Screen',
      'log4perl.appender.CONSOLE.layout' => 'Log::Log4perl::Layout::PatternLayout',
      'log4perl.appender.CONSOLE.layout.ConversionPattern' => '%d [%p] %m %n'}
     };
my $LOGGER;


main(@ARGV);

sub main {
    my (@del_files_lists) = @_;

    for my $del_files_list ( @del_files_lists ){
        # $del_files_list のfileには削除対象のobj keyを改行区切りで記載
        $LOGGER = init_logger($del_files_list);

        # 各削除file一覧は、chunk_size_limit 毎に分割して読込みます
        my $del_files_chunks = load_del_files_list($del_files_list);

        my $i = 0;
        for my $del_files_chunk ( @$del_files_chunks ){
            my $j = 0;
            $i++;
            for my $obj_key_org ( @$del_files_chunk ){
                next unless $obj_key_org;
                
                $j++;
                $LOGGER->info("$i $j $obj_key_org");
                
                # url encoding
                my $obj_key = $obj_key_org;
                $obj_key =~ s/([^ 0-9a-zA-Z])/"%".uc(unpack("H2",$1))/eg;
                
                # version id取得
                my $verion_id = get_version_id($obj_key);
                if( $verion_id ){
                    $LOGGER->info("get_version_id($obj_key_org) -> $verion_id");
                } else {
                    $LOGGER->error("fail get_version_id($obj_key_org)");
                    next;
                }

            }

            sleep( $CONF->{chunk_interval} );

            last;  # for debug
        }
    }
}

sub load_del_files_list {
    my ($del_files_list) = @_;

    open my $fh,"<", $del_files_list or
        die "fail open $del_files_list $!";
    my $ret = [];
    my $chunk_no = 0;
    my $chunk_size = 0;
    for my $line ( <$fh> ){
        chomp($line);
        if( not defined($ret->[$chunk_no]) ){
            $ret->[$chunk_no] = [];
        }
        if($chunk_size < $CONF->{chunk_size_limit}){
            push(@{$ret->[$chunk_no]}, $line);
            $chunk_size++;
        } else {
            $chunk_size = 0;
            $chunk_no++;
        }
    }
    close($fh);

    return $ret;
}


sub del_s3_obj {
    my ($obj_key, $verion_id) = @_;

    $verion_id = '' unless $verion_id;
    
    my $url = join('/',
                   "http://$CONF->{aws_s3}->{host}",
                   "$CONF->{aws_s3}->{bucket}",
                   $obj_key);
   $url .= "?versionId=$verion_id";

    my $date_str = time2str();
    my $str_to_sign_org =
        join("\n",
             'DELETE',
             '',
             '',
             $date_str,
             "/$CONF->{aws_s3}->{bucket}/$obj_key");
    $str_to_sign_org .= "?versionId=$verion_id";

    my $hmac = Digest::HMAC_SHA1->new($CONF->{aws_s3}->{aws_secret_access_key});
    $hmac->add($str_to_sign_org);
    my $sig = $hmac->b64digest . '=';
    
    my $header =
        [
         'Host' => $CONF->{aws_s3}->{host},
         'Date'=> $date_str,
         'Authorization'=>
         "AWS $CONF->{aws_s3}->{aws_access_key_id}:$sig"];
    
    my $req = HTTP::Request->new('DELETE', $url, $header);
    my $ua = LWP::UserAgent->new;
    my $res = $ua->request($req);

    if(not $res->is_success) {
        $LOGGER->error( $res->status_line );
        $LOGGER->error( Dumper($res) );
        return;
    }
    return $obj_key;
}


sub get_version_id {
    my ($obj_key) = @_;
    
    my $url = join('/',
                   "http://$CONF->{aws_s3}->{host}",
                   "$CONF->{aws_s3}->{bucket}",
                   $obj_key);
    my $date_str = time2str();
    my $str_to_sign_org =
        join("\n",
             'HEAD',
             '',
             '',
             $date_str,
             "/$CONF->{aws_s3}->{bucket}/$obj_key");

    my $hmac = Digest::HMAC_SHA1->new($CONF->{aws_s3}->{aws_secret_access_key});
    $hmac->add($str_to_sign_org);
    my $sig = $hmac->b64digest . '=';
    
    my $header =
        [
         'Host' => $CONF->{aws_s3}->{host},
         'Date'=> $date_str,
         'Authorization'=>
         "AWS $CONF->{aws_s3}->{aws_access_key_id}:$sig"];
    
    my $req = HTTP::Request->new('HEAD', $url, $header);
    my $ua = LWP::UserAgent->new;
    my $res = $ua->request($req);

    return $res->header('x-amz-version-id');
}

sub init_logger {
    my ($org_filepaths_list) = @_;

    if ($org_filepaths_list){
        my($filename, $dirs) = File::Basename::fileparse($org_filepaths_list);
        my $datetime = unixtime_to_str(time);
        $CONF->{log}->{'log4perl.appender.LOGFILE.filename'} =
            $CONF->{log}->{'log4perl.appender.LOGFILE.dir'} ."/".
            join(".",$filename,$datetime,"log");
    }
    
    Log::Log4perl::init($CONF->{log});
    my $logger = Log::Log4perl::get_logger("rootLogger");
    unless($logger){
        die "fail init_logger() $!";
    }
    return $logger;
}

sub unixtime_to_str {
    my ($unix_time) = @_;

    my ($sec,$min,$hour, $mday,$mon,$year,$wday,$yday,$isdst) =
        localtime($unix_time);
    return sprintf("%04d%02d%02dT%02d%02d%02d",
                   1900+$year,$mon+1,$mday,$hour,$min,$sec);
}

1;

Net::Amazon::S3 for perl で、aws s3にあるファイル一覧を取得

https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html

aws s3の仕様は↑こちらで、 これを Net::Amazon::S3 for perl で実装したものが、以下。

大量のファイルでもあっても取得できるポイントは、 perl src内にあるmarkとnext_maker で、 これがページングの役割を担っています。

#!/usr/local/bin/perl
use strict;
use warnings;
use Encode;
use Net::Amazon::S3;
use Data::Dumper;

my $CONF =
    {aws_s3=>
     {client=>{
               host=>                   'ないしょ',
               aws_access_key_id =>     'ないしょ',
               aws_secret_access_key => 'ないしょ',
               retry => 1},
      bucket=>'ないしょ'},
     
     ls=>{max_keys=>5,     # 一度に取得するobj keyの制限
          delimiter=>"/",  # 不要と思いますが、これがないとs3がis_truncatedを返さない...
          max_exec=>10,    # lsを実行する回数制限
          sleep=>2}        # やさしく ls
    };


main( @ARGV );

sub main {
    my ($kyoten_code) = @_;

    unless( $kyoten_code ){
        return disp_usage();
    }

    my $s3 = Net::Amazon::S3->new(%{$CONF->{aws_s3}->{client}});
    $s3->ua( LWP::UserAgent->new(ssl_opts=>{verify_hostname =>0}) );

    my $bucket = $s3->bucket($CONF->{aws_s3}->{bucket} );

    my $list_opt =
        {prefix=>    $kyoten_code,
         max_keys => $CONF->{ls}->{max_keys},
         delimiter=> $CONF->{ls}->{delimiter} };
    my $pre_ls_res = {is_truncated=>1};

    my $ls_exec_count = 0;
    my $obj_keys_size = 0;
    
    while($pre_ls_res->{is_truncated} and
          $ls_exec_count++ < $CONF->{ls}->{max_exec} ){

        # 前回の続きから、ls する為、markerをset
        $list_opt->{marker} = $pre_ls_res->{next_marker};

        my $ls_res = $bucket->list($list_opt);
        
        $obj_keys_size += scalar(@{$ls_res->{keys}});
        
        for my $obj_info ( @{$ls_res->{keys}} ){
            # version idは別途、取得する必要あり
            my $obj_detail = $bucket->head_key($obj_info->{key});
            
            my $disp_str = join("\t",
                                $obj_info->{key},
                                $obj_detail->{'x-amz-version-id'} || '',
                                $obj_info->{size},
                                $obj_info->{etag},
                                $obj_info->{last_modified});
            print "$disp_str\n";
        }

        # 続きがある場合、is_truncated = true ですって
        $pre_ls_res->{is_truncated} = $ls_res->{is_truncated};
        $pre_ls_res->{next_marker} = $ls_res->{next_marker};

        sleep($CONF->{ls}->{sleep});
    }

    print "You have $obj_keys_size objects !!\n";
}

sub disp_usage {
    print "Usage: $0 HEIM_KYOTEN_CODE.\n";
}


1;

IT資産(ハードウェアやソフトウェア)を管理するDB table構成を考える - CMDB

↓こんな感じでOKかと思います

CREATE DATABASE it_assets CHARACTER SET utf8;
grant all privileges on it_assets.* to ないしょ@'%' identified by 'ないしょ';

CREATE TABLE data_center_doc (
id                      int AUTO_INCREMENT,
doc_name                varchar(100) not null unique comment 'DC文書名 or サービス名',
location                varchar(30)    comment '設置場所. 西日本, AWS',
data_center_doc_url     varchar(200)    comment 'DC文書の保管先url',
note                    varchar(300)    comment '概要',
dc_usage_price          int             comment 'DC利用料 円/月',
primary key(id)
)
COMMENT='データセンタA文書の単位で記載';

CREATE TABLE service (
dc_doc_name             varchar(100)   comment 'tbl:data_center_docのname',
id                      int AUTO_INCREMENT,
service_name            varchar(100) not null unique comment 'サービス名',
note                    varchar(300)    comment '概要',
monitor_sys_url         varchar(200)    comment '例:xymon画面のurl',
service_start_date      date            coment 'サービス開始日',
sla_price               int             comment '運用費 円/月',
sla_start_date          date            comment '運用契約 開始日',
sla_doc                 varchar(200)    comment '運用契約書の保管先url',
owner_dept              varchar(50)     comment 'オーナ部署',
owner_man               varchar(50)     comment 'オーナ部署 責任者、担当者',
our_sales_dept          varchar(50)     comment '自社 営業 部署',
our_sales_man           varchar(50)     comment '自社 営業 責任者、担当者',
our_dev_dept            varchar(50)     comment '自社 開発 部署',
our_dev_man             varchar(50)     comment '自社 開発 責任者、担当者',
our_ops_dept            varchar(50)     comment '自社 運用 部署',
our_ops_man             varchar(50)     comment '自社 運用 責任者、担当者',
primary key(id)
)
COMMENT='運用契約の単位で記載';

ALTER TABLE service ADD CONSTRAINT service_dc_doc_name
FOREIGN KEY (dc_doc_name) REFERENCES data_center_doc (doc_name)
ON DELETE SET NULL ON UPDATE CASCADE;


CREATE TABLE service_option (
dc_doc_name             varchar(100)   comment 'tbl:data_center_docのname',
id                      int AUTO_INCREMENT,
type                    varchar(10)     comment 'ssl, dns, other',
name                    varchar(100) not null comment '名称',
note                    varchar(300)    comment '概要',
running_price           int             comment 'ランニング費 円/月',
vender                  varchar(50)     comment 'ベンダー',
start_date              date            comment '開始日',
nend_date               date            comment '終了日',
primary key(id)
)
COMMENT='dnsやssl証明書 等. dnsやsslの有効期限はscriptで自動checkしたい';

ALTER TABLE service_option ADD CONSTRAINT service_option_dc_doc_name
FOREIGN KEY (dc_doc_name) REFERENCES data_center_doc (doc_name)
ON DELETE SET NULL ON UPDATE CASCADE;


CREATE TABLE hardware (
dc_doc_name             varchar(100)   comment 'tbl:data_center_docのname',
id                      int AUTO_INCREMENT,
hw_type                 varchar(10)  not null comment 'オンプレ 統合WIN 統合DB AWS ',
hw_name                 varchar(100) not null comment '名称',
maker                   varchar(50),
model                   varchar(80)     comment '型番',
quantity                int default 1   comment '数量. 冗長化の場合、2以上',
note                    varchar(300)    comment '概要',
support_vender          varchar(50)     comment '保守業者',
support_start_date      date            comment '保守 開始日',
support_end_date        date            comment '保守 終了日',
extra_support_end_date  date            comment '特別 終了日',
support_end_note        varchar(300)    comment 'EOLのアナウンス有無等',
primary key(id)
)
COMMENT='サーバやロードバランサ、外付けドライブ 等';

ALTER TABLE hardware ADD CONSTRAINT hardware_dc_doc_name
FOREIGN KEY (dc_doc_name) REFERENCES data_center_doc (doc_name)
ON DELETE SET NULL ON UPDATE CASCADE;

-- 外部参照keyを貼られる側には、indexが必要みたい
create index hardware_hw_name on hardware(hw_name);


CREATE TABLE software (
dc_doc_name             varchar(100) comment 'tbl:data_center_docのname',
hw_name                 varchar(100) comment 'tbl:hardwareのhw_name',
id                      int AUTO_INCREMENT,
sw_type                 varchar(10)  not null comment 'OS MW AP OTHER',
sw_name                 varchar(100) not null,
maker                   varchar(50),
model                   varchar(200)     comment '型番 ex. uname -a. perl -v',
quantity                int default 1   comment 'ライセンス数',
note                    varchar(300)    comment '概要',
support_vender          varchar(50)     comment '保守業者',
support_start_date      date            comment '保守 開始日',
support_end_date        date            comment '保守 終了日',
extra_support_end_date  date            comment '特別 終了日',
support_end_note        varchar(300)    comment 'EOLのアナウンス有無等',
primary key(id)
)
COMMENT='OSやミドルウェア 等. 各swのversionはxymon経由で自動収集したい';

ALTER TABLE software ADD CONSTRAINT software_dc_doc_name
FOREIGN KEY (dc_doc_name) REFERENCES data_center_doc (doc_name)
ON DELETE SET NULL ON UPDATE CASCADE;

ALTER TABLE software ADD CONSTRAINT software_hw_name
FOREIGN KEY (hw_name) REFERENCES hardware (hw_name)
ON DELETE SET NULL ON UPDATE CASCADE;

ALTER TABLE software ADD CONSTRAINT software_hw_name
FOREIGN KEY (hw_name) REFERENCES hardware (hw_name)
ON DELETE SET NULL ON UPDATE CASCADE;

perlで、日付&時刻をparseする

https://metacpan.org/release/Date-Calc

perlで日付や時刻を扱う場合、Date::Calc を使用していますが、 2015/3以降、メンテナンスが止まっているようです。

殆どの場合、Date::Calc で足りていますが、 日付&時刻な文字列をparseする必要がある場合、困ることがあります。

そんなときは...、「DateTime::Format::~ 」シリーズとして、

  • DateTime::Format::Strptime
  • DateTime::Format::HTTP
  • DateTime::Format::Mail
  • DateTime::Format::Japanese

があります。

これを機会に Date::Calc → DateTime へ移行した方がよいのかもしれませんね。

以下、ざっと書いたサンプル

#!/usr/local/bin/perl
use strict;
use warnings;
use Data::Dumper;
use DateTime::Format::Strptime;
use DateTime::Format::HTTP;
use Data::Dumper;

my $TIME_OFFSET_MIN = 1;

main();

sub main {
    print "TIME: ",time(),"\n";
    
    
    my $str_1 = '2019/01/18 18:02:22';

    my $parser_1 =
        DateTime::Format::Strptime->new(pattern=>"%Y/%m/%j %H:%M:%S",
                                        time_zone=>'Asia/Tokyo');
    my $date_1 = $parser_1->parse_datetime($str_1);
    $date_1->add(minutes => $TIME_OFFSET_MIN);
#    print Dumper($date_1);
    print $date_1->datetime,"\n";
    
    my $str_2 = 'Fri, 18 Jan 2019 10:02:22 GMT';
    my $date_2 = DateTime::Format::HTTP->parse_datetime($str_2);
    print $date_2->datetime,"\n";

    # 時刻比較
    print DateTime->compare( $date_1, $date_2 ), "\n";
}

ちなみに、unix timeからの DateTime オブジェクト作成は以下

    my $date_nas = DateTime->from_epoch( epoch =>$unix_time);

VirtualBoxのNW設定 ( NATネットワーク + ホストオンリーアダプタ )

新規にCentOS7のゲストOSを追加しようとしましたが、 NW周りの設定方法をスッカリ忘れていたので、メモ。

以下は正しい設定という訳ではありません。「とりあえずは動作している」程度です。

VirtualBoxにおけるゲストOS 設定

f:id:end0tknr:20190113070543p:plain

f:id:end0tknr:20190113070545p:plain

f:id:end0tknr:20190113070642p:plain

VirtualBoxにおけるホストオンリーアダプタ 設定

メニューバー → ファイル → ホストネットワークマネージャ

f:id:end0tknr:20190113070739p:plain

f:id:end0tknr:20190113070731p:plain

centos7側

NATネットワーク側

$ sudo cat /etc/sysconfig/network-scripts/ifcfg-enp0s3
TYPE="Ethernet"
#PROXY_METHOD="none"
#BROWSER_ONLY="no"
BOOTPROTO="dhcp"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="enp0s3"
UUID="e9670a5a-1f9d-456c-862d-0e912a87bf42"
DEVICE="enp0s3"
ONBOOT="yes"
PERDNS=yes
PEERROUTES=yes

ホストオンリーアダプタ側

$ sudo cat /etc/sysconfig/network-scripts/ifcfg-enp0s8
TYPE="Ethernet"
PROXY_METHOD="none"
#BROWSER_ONLY="no"
#BOOTPROTO="dhcp"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="no"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
#IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="enp0s8"
UUID="e9670a5a-1f9d-456c-862d-0e912a87bf42"
DEVICE="enp0s8"
ONBOOT="yes"
IPADDR=192.168.244.101
#ZONE=

その他

selinuxfirewallを確認する

o/rマッパー (というより、sqlマッパー) MyBatis for java

sqlを多く書く為、java用o/rマッパーとして、Doma2 or MyBatis を検討。

先程のエントリの通り、Doma2は諦めた為、MyBatis をお試し。

TODO

DBへの Service→Dao→Entity 経由でのアクセスは理解できました。 が、これまで new()したobjectを利用するsourceを多く書いてきたので、 DIな構成を十分に理解できていない...

環境

項目 内容
OS windows10
java C:\Program Files\Java\jdk-10.0.1
IDE Eclipse Java EE ver.Oxygen(4.7.3a) Build:201804

dir 構成 (主要部のみ抜粋)

[SpringBootMyBatis]$ tree
├─ build.gradle
└─ src
    └─ main
         ├─ java
         │   └─ jp
         │       └─ end0tknr
         │           ├─ controller
         │           │   └─ Controller.java
         │           ├─ domain
         │           │   ├─ UserDao.java
         │           │   ├─ UserEntity.java
         │           │   └─ UserService.java
         │           └── SpringBootMyBatisApplication.java
         └── resources
             ├─ application.yml
             └─ jp
                 └─ end0tknr
                     └─ domain
                         └─ UserDao.xml  

file内容

build.gradle

ubuildscript {
    ext {
        springBootVersion = '2.1.1.RELEASE'
    }
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
    }
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

group = 'jp.end0tknr'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    implementation('org.springframework.boot:spring-boot-starter-web')
    implementation('org.mybatis.spring.boot:mybatis-spring-boot-starter:1.3.2')
    runtimeOnly('mysql:mysql-connector-java')
    compileOnly('org.projectlombok:lombok')
    testImplementation('org.springframework.boot:spring-boot-starter-test')
}

src/main/resources/application.yml

spring:
  datasource:
    url: jdbc:mysql://192.168.244.101:3306/xing?useSSL=false&zeroDateTimeBehavior=convertToNull
    username: ????????
    password: ????????
    driver-class-name: com.mysql.cj.jdbc.Driver

src/main/java/jp/end0tknr/controller/Controller.java

package jp.end0tknr.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.util.Arrays;
import java.util.List;

import org.apache.ibatis.session.SqlSession;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import jp.end0tknr.domain.UserDao;
import jp.end0tknr.domain.UserEntity;
import jp.end0tknr.domain.UserService;

@RestController
public class Controller {
    private static final Logger logger =
            LoggerFactory.getLogger(Controller.class);

    @Autowired
    private UserDao userDao;
    @Autowired
    private SqlSession sqlSession;

    @Autowired
    private UserService userService;
    
    //この部分は、MyBatisと無関係です
    @RequestMapping("/")
    public String index() {
        logger.info("start index()");
        return "This is index page !!";
    }

    // DAO経由でレコード取得。SQLはjava内に記載
    @RequestMapping("/dumpuser1")
    public String dumpUser1() {
        UserEntity userEntity = userDao.findByUid("endou021");
        return "dumpUser1() is ..." + userEntity.toString();
        
    }

    // SqlSession経由でレコード取得。SQLは外部XML内に記載
    @RequestMapping("/dumpuser2")
    public String dumpUser2() {
        UserEntity userEntity = 
                sqlSession.selectOne("jp.end0tknr.domain.UserDao.findByUid2","endou021");

        return "dumpUser2() is ..." + userEntity.toString();
    }

    // Service→DAO経由でレコード取得。SQLはjava内に記載
    // 単にobjectをreturnしていますが、自動的にjsonになります
    @RequestMapping("/dumpusers1")
    public List<UserEntity> dumpUsers1() {
        List<UserEntity> userEntities = userService.findAll();

        return userEntities; 
    }

    // 動的SQLパターン
    @RequestMapping("/dumpusers2")
    public List<UserEntity> dumpUsers2() {
        List<UserEntity> userEntities =
                userService.findByUids(Arrays.asList("dada018","undou099"));
        return userEntities; 
    }
}

src/main/java/jp/end0tknr/domain/UserEntity.java

package jp.end0tknr.domain;

import lombok.Data;

@Data
public class UserEntity {
    private String uid;
    private String name;
    private String bushoId;
    
    @Override
    public String toString() {
        return getUid() + "," + getName() + "," + getBushoId();
    }
}

src/main/java/jp/end0tknr/domain/UserDao.java

package jp.end0tknr.domain;

import java.util.ArrayList;
import java.util.List;

import org.apache.ibatis.annotations.Mapper;
import org.apache.ibatis.annotations.Param;
import org.apache.ibatis.annotations.Select;
import org.apache.ibatis.annotations.SelectProvider;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import jp.end0tknr.controller.Controller;

@Mapper
public interface UserDao {
    static Logger logger =
            LoggerFactory.getLogger(Controller.class);

    @Select("SELECT uid, name, busho_id FROM user WHERE uid = #{uid}")
    UserEntity findByUid(@Param("uid") String uid);

    UserEntity findByUid2(@Param("uid") String uid);

    @Select("SELECT * FROM user order by uid")
    List<UserEntity> findAll();

    // 動的SQL
    @SelectProvider(type = SqlProvider.class, method = "findByUids")
    List<UserEntity> findByUids(@Param("uids") List<String> uids);

    static final class SqlProvider {
        public SqlProvider() {}
        public String findByUids(@Param("uids") List<String> uids) {
            StringBuilder sqlBuilder =
                    new StringBuilder("SELECT uid, name FROM user where uid IN (");
            
            
            ArrayList<String> uidsTmp = new ArrayList<>();
            for (String uid : uids) { 
                uidsTmp.add("'"+ uid + "'"); //TODO sql sanitize
            }
            sqlBuilder.append(String.join(",", uidsTmp) + ")");

            String sqlStr = sqlBuilder.toString();
            logger.info(sqlStr);
            
            return sqlBuilder.toString();
        }
    }    
    
    
}

src/main/java/jp/end0tknr/domain/UserService.java

package jp.end0tknr.domain;

import java.util.List;

import org.apache.ibatis.session.SqlSession;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class UserService {
    @Autowired
    private UserDao userDao;

    public List<UserEntity> findAll() {
        return userDao.findAll();
    }    
    
    public List<UserEntity> findByUids(List<String> uids) {
        return userDao.findByUids(uids);
    }
}

src/main/resources/jp/end0tknr/domain/UserDao.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="jp.end0tknr.domain.UserDao">
  <select id="findByUid2" resultType="jp.end0tknr.domain.UserEntity">
    SELECT uid, name, busho_id FROM user WHERE uid = #{uid}
  </select>
</mapper>

o/rマッパー doma2 for java の環境構築...を諦めた

2way-sqlを試したくて、↑この辺りを参考に、環境構築を進めましたが、以下のエラーが解消できない。 (エラーの内容は、このエントリーの下部参照)。

今後、改めて調べるかもしれないので、現状をメモとして記載。

dir 構成

win環境から、linux環境へcopyし、treeコマンドで表示しています

[SpringBootDoma]
├ bin
│ ├ default
│ ├ main
│ │ ├ application.yml
│ │ ├ jp
│ │ │ └ end0tknr
│ │ │     ├ controller
│ │ │     │ └ Controller.class
│ │ │     ├ domain
│ │ │     │ ├ UserDao.class
│ │ │     │ ├ UserEntity.class
│ │ │     │ └ UserService.class
│ │ │     └ SpringBootDomaApplication.class
│ │ ├ META-INF
│ │ │ └ jp
│ │ │     └ end0tknr
│ │ │         └ domain
│ │ │             └ UserDao
│ │ │                 └ selectAll.sql
│ │ └ templates
│ └ test
│     └ jp
│         └ end0tknr
│             └ SpringBootDomaApplicationTests.class
├ build.gradle
├ gradle
│ └ wrapper
│     ├ gradle-wrapper.jar
│     └ gradle-wrapper.properties
├ gradlew
├ gradlew.bat
├ settings.gradle
└ src
    ├ main
    │ ├ java
    │ │ └ jp
    │ │     └ end0tknr
    │ │         ├ controller
    │ │         │ └ Controller.java
    │ │         ├ domain
    │ │         │ ├ UserDao.java
    │ │         │ ├ UserEntity.java
    │ │         │ └ UserService.java
    │ │         └ SpringBootDomaApplication.java
    │ └ resources
    │     ├ application.yml
    │     ├ META-INF
    │     │ └ jp
    │     │     └ end0tknr
    │     │         └ domain
    │     │             └ UserDao
    │     │                 └ selectAll.sql
    │     ├ static
    │     └ templates
    └ test
        └ java
            └ jp
                └ end0tknr
                    └ SpringBootDomaApplicationTests.java

主要fileの内容

build.gradle

buildscript {
    ext {
        springBootVersion = '2.1.1.RELEASE'
    }
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
    }
}

apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

group = 'jp.end0tknr'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

// for Doma 2
// JavaクラスとSQLファイルの出力先ディレクトリを同じにする
processResources.destinationDir = compileJava.destinationDir
// コンパイルより前にSQLファイルを出力先ディレクトリにコピーするために依存関係を逆転する
compileJava.dependsOn processResources

dependencies {
    implementation('org.springframework.boot:spring-boot-starter-web')
    runtimeOnly('mysql:mysql-connector-java')
    compileOnly('org.projectlombok:lombok')
    testImplementation('org.springframework.boot:spring-boot-starter-test')

    // doma exclude springframework
    annotationProcessor("org.seasar.doma.boot:doma-spring-boot-starter:1.1.1")
    compile("org.seasar.doma.boot:doma-spring-boot-starter:1.1.1") {
            exclude group: "org.springframework.boot"
    
    }
}

src/main/resources/application.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.244.101:3306/xing?useSSL=false&zeroDateTimeBehavior=convertToNull
    username: ???????
    password: ???????
    driver-class-name: com.mysql.cj.jdbc.Driver
doma:
  dialect: mysql
  sql-file-repository: NO_CACHE

jp.end0tknr.controller.Controller.java

package jp.end0tknr.controller;

import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import jp.end0tknr.domain.UserEntity;
import jp.end0tknr.domain.UserService;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@RestController
public class Controller {
    private static final Logger logger =
            LoggerFactory.getLogger(Controller.class);
    @Autowired
    UserService userService;
    
    @RequestMapping("/")
    public String index() {
        return "Hello Spring Boot !!";
    }

    @RequestMapping("/dumpusers")
    public List<UserEntity> dumpUsers() {
        List<UserEntity> list = userService.selectAll();
        return list;
    }
}

jp.end0tknr.domain.UserEntity.java

package jp.end0tknr.domain;

import org.seasar.doma.Entity;
import org.seasar.doma.Id;
import org.seasar.doma.Table;
import lombok.Data;

@Data
@Entity
@Table(name="user")
public class UserEntity {
    @Id
    private String uid;
    private String name;
    private String bushoId;
}

jp.end0tknr.domain.UserDao.java

package jp.end0tknr.domain;

import org.seasar.doma.Entity;
import org.seasar.doma.Id;
import org.seasar.doma.Table;
import lombok.Data;

@Data
@Entity
@Table(name="user")
public class UserEntity {
    @Id
    private String uid;
    private String name;
    private String bushoId;
}

jp.end0tknr.domain.UserService.java

package jp.end0tknr.domain;

import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class UserService {
    @Autowired
    UserDao userDao;

    public List<UserEntity> selectAll() {
        return userDao.selectAll();
    }
}

src/main/resources/META-INF/jp/end0tknr/domain/UserDao/selectAll.sql

select uid,name,busho_id from user order by uid;

表示されたエラー

Description:
Field in required a bean of type that could not be found  org.springframework.beans.factory.annotation.Autowired(required=true)

The injection point has the following annotations:
    - @org.springframework.beans.factory.annotation.Autowired(required=true)
Action:
Consider defining a bean of type 'jp.end0tknr.domain.UserDao' in your configuration.

↑こちらや↓こちら。いつからか、上記エラーを再現できず、下記エラーばかり発生...

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.1.1.RELEASE)

2019-01-04 08:43:56.692  INFO 16124 --- [           main] jp.end0tknr.SpringBootDomaApplication    : Starting SpringBootDomaApplication on LAPTOP-Q767CC9I with PID 16124 (C:\home\end0tknr\eclipse-workspace-oxygen\SpringBootDoma\bin\main started by end0t in C:\home\end0tknr\eclipse-workspace-oxygen\SpringBootDoma)
2019-01-04 08:43:56.695  INFO 16124 --- [           main] jp.end0tknr.SpringBootDomaApplication    : No active profile set, falling back to default profiles: default
2019-01-04 08:43:57.221 ERROR 16124 --- [           main] o.s.boot.SpringApplication               : Application run failed

java.lang.IllegalStateException: Error processing condition on org.seasar.doma.boot.autoconfigure.DomaAutoConfiguration.sqlFileRepository
    at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:64) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForBeanMethod(ConfigurationClassBeanDefinitionReader.java:181) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:141) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:117) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:327) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:232) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:275) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:95) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:691) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:528) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:142) ~[spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:775) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:316) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at jp.end0tknr.SpringBootDomaApplication.main(SpringBootDomaApplication.java:10) [main/:na]
Caused by: java.lang.IllegalStateException: Failed to introspect Class [org.seasar.doma.boot.autoconfigure.DomaAutoConfiguration] from ClassLoader [sun.misc.Launcher$AppClassLoader@764c12b6]
    at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:686) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:583) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:568) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:626) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1688) ~[na:1.8.0_191]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:721) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:662) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:630) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1518) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1023) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.addBeanTypeForNonAliasDefinition(BeanTypeRegistry.java:195) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.addBeanTypeForNonAliasDefinition(BeanTypeRegistry.java:159) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.addBeanType(BeanTypeRegistry.java:152) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.updateTypesIfNecessary(BeanTypeRegistry.java:140) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[na:1.8.0_191]
    at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.updateTypesIfNecessary(BeanTypeRegistry.java:135) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.BeanTypeRegistry.getNamesForType(BeanTypeRegistry.java:97) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.OnBeanCondition.collectBeanNamesForType(OnBeanCondition.java:298) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getBeanNamesForType(OnBeanCondition.java:289) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getBeanNamesForType(OnBeanCondition.java:278) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getMatchingBeans(OnBeanCondition.java:189) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.OnBeanCondition.getMatchOutcome(OnBeanCondition.java:160) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:47) ~[spring-boot-autoconfigure-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    ... 17 common frames omitted
Caused by: java.lang.NoClassDefFoundError: org/springframework/dao/support/PersistenceExceptionTranslator
    at java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_191]
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_191]
    at java.lang.Class.getDeclaredMethods(Class.java:1975) ~[na:1.8.0_191]
    at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:668) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    ... 39 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.springframework.dao.support.PersistenceExceptionTranslator
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[na:1.8.0_191]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_191]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[na:1.8.0_191]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_191]
    ... 43 common frames omitted

2019-01-04 08:43:57.227  WARN 16124 --- [           main] o.s.boot.SpringApplication               : Unable to close ApplicationContext

java.lang.IllegalStateException: Failed to introspect Class [org.seasar.doma.boot.autoconfigure.DomaAutoConfiguration] from ClassLoader [sun.misc.Launcher$AppClassLoader@764c12b6]
    at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:686) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:583) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:568) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.util.ReflectionUtils.getUniqueDeclaredMethods(ReflectionUtils.java:626) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1688) ~[na:1.8.0_191]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getTypeForFactoryMethod(AbstractAutowireCapableBeanFactory.java:721) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.determineTargetType(AbstractAutowireCapableBeanFactory.java:662) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:630) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.isFactoryBean(AbstractBeanFactory.java:1518) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doGetBeanNamesForType(DefaultListableBeanFactory.java:507) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeanNamesForType(DefaultListableBeanFactory.java:477) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:598) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.getBeansOfType(DefaultListableBeanFactory.java:590) ~[spring-beans-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.getBeansOfType(AbstractApplicationContext.java:1204) ~[spring-context-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    at org.springframework.boot.SpringApplication.getExitCodeFromMappedException(SpringApplication.java:905) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.getExitCodeFromException(SpringApplication.java:891) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.handleExitCode(SpringApplication.java:877) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:826) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:327) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1248) [spring-boot-2.1.1.RELEASE.jar:2.1.1.RELEASE]
    at jp.end0tknr.SpringBootDomaApplication.main(SpringBootDomaApplication.java:10) [main/:na]
Caused by: java.lang.NoClassDefFoundError: org/springframework/dao/support/PersistenceExceptionTranslator
    at java.lang.Class.getDeclaredMethods0(Native Method) ~[na:1.8.0_191]
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) ~[na:1.8.0_191]
    at java.lang.Class.getDeclaredMethods(Class.java:1975) ~[na:1.8.0_191]
    at org.springframework.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:668) ~[spring-core-5.1.3.RELEASE.jar:5.1.3.RELEASE]
    ... 21 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.springframework.dao.support.PersistenceExceptionTranslator
    at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[na:1.8.0_191]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_191]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[na:1.8.0_191]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_191]
    ... 25 common frames omitted

Spring Boot for java で yamlな設定ファイル( application.yml )を参照

https://end0tknr.hateblo.jp/entry/20180114/1515907344

以前の上記エントリーの再編集でポイントは以下の通り。

  • aplication.properties の代わりに application.yml があると、Spring Boot が自動認識
  • snake caseで記載しても、camel case として認識
  • 今回、記載していませんが、環境変数:SPRING_PROFILES_ACTIVE=dev 等により application-dev.yml のような設定に切り替え可能

参考url

https://www.saka-en.com/java/spring-boot-read-yaml/

https://www.saka-en.com/java/spring-boot-get-yaml-value/

サンプルアプリの構成

今回は、 application.yml の内容を CommonConf.java で読み、 Controller1.javajson形式で出力(といっても単なるdump)します。

src/main/java/
  jp.end0tknr/       ---1)
    SpringBootDomaApplication.java ---2)
    Controller1.java ---3)
  jp.end0tknr.conf/  ---4)
    CommonConf.java  ---5)   
src/main/resources/  ---6)
  application.yml    ---7)
build.gradle         ---8)  
  • 1),2) : spring bootがproject追加時に自動作成 (私は何もしていません)
  • 3) : 設定fileの内容をjsonで返すrest controller
  • 4),5) : 設定fileへのアクセサ?
  • 6),7) : 設定file
  • 8) : 特別な変更は行っていませんが、参考として build.gradle を記載

application.yml

以下のようにサンプルを用意しました。 spring.datasource.~ は、value を変更することでmysqlへも接続できます。

spring:
  datasource:
    url: jdbc:mysql://192.168.244.101:3306/xing?useSSL=false&zeroDateTimeBehavior=convertToNull
    username: xingadm
    password: xxxxxxx
    driver-class-name: com.mysql.cj.jdbc.Driver
common:
  app_root_url: https://www.google.co.jp
  login_url: https://www.yahoo.co.jp
  news_list:
    - top
    - tech
    - economy
    - weather

CommonConf.java

application.yml を読むためのもので、ポイントは「@Component」と 「@ConfigurationProperties(prefix="common")」。

「@Data」は、lombok によるアクセサメソッド自動生成の為のものです。

package jp.end0tknr.conf;

import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import lombok.Data;

@Component
@ConfigurationProperties(prefix="common")
@Data
public class CommonConf {
    private String appRootUrl;
    private String loginUrl;
    private String[] newsList;
}

Controller1.java

設定内容の渡し方は様々ですが、今回は、commonConf に @Autowired し、 commonConf をそのまま return することで、json 形式で返しています。

package jp.end0tknr;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import jp.end0tknr.conf.CommonConf;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@RestController
public class Controller1 {

    private static final Logger logger =
            LoggerFactory.getLogger(Controller1.class);

    @Autowired
    private CommonConf commonConf;
    
    @RequestMapping("/")
    public String index() {
        return "Hello Spring Boot !!" + commonConf.getAppRootUrl();
    }

    @RequestMapping("/dumpconf")
    public CommonConf dumpConf() {
        return commonConf;
    }

}```


# dump結果

ブラウザで、http://localhost:8080/dumpconf へアクセスすると、
次のように表示されます。

** application.yml では、snakeで記載しましたが、camelで表示されています。 **

{"appRootUrl":"https://www.google.co.jp", "loginUrl":"https://www.yahoo.co.jp", "newsList":["top","tech","economy","weather"]}

※ 視認性を良くする為、上記には改行を入れ、整形しています **

Class::Accessor for perl の java版 Lombok である eclipse + gradle な環境へ導入

久しぶりに eclipseを触ったら、すっかり忘れていたので、メモ。

今更、getter() / setter()を大量に書きたくありませんからね。

STEP1 : eclipse への導入

https://projectlombok.org/downloads/lombok.jar をダウンロードし、lombok.jar を実行するだけ。

自身のPCにあるIDE (eclipse)を探し出し、 lombok.jar 導入だけでなく、eclipse.ini の編集まで自動で行ってくれます。

以下、lombok.jar 実行後のインストーラの画面と、インストール後の eclipse.ini

f:id:end0tknr:20181229180427p:plain

-startup
plugins/org.eclipse.equinox.launcher_1.5.200.v20180922-1751.jar
--launcher.library
C:\Users\end0t\.p2\pool\plugins\org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.900.v20180922-1751
-product
org.eclipse.epp.package.jee.product
-showsplash
org.eclipse.epp.package.common
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.8
-Dosgi.instance.area.default=@user.home/eclipse-workspace
-XX:+UseG1GC
-XX:+UseStringDeduplication
--add-modules=ALL-SYSTEM
-Dosgi.requiredJavaVersion=1.8
-Dosgi.dataAreaRequiresExplicitInit=true
-Xms256m
-Xmx1024m
--add-modules=ALL-SYSTEM
-Declipse.p2.max.threads=10
-Doomph.update.url=http://download.eclipse.org/oomph/updates/milestone/latest
-Doomph.redirection.index.redirection=index:/->http://git.eclipse.org/c/oomph/org.eclipse.oomph.git/plain/setups/
-javaagent:C:\eclipse\jee-2018-12_v4.9\eclipse\lombok.jar

STEP2 : gradleなprojectへのlombak 導入

build.gradle に 「apply plugin: 'war'」と 「 providedCompile('org.projectlombok:lombok:1.18.4')」 を追加し、 projectの右クリックから「Gradle > Reflesh Gradle Project」を実行するだけ。

以下、build.gradle

buildscript {
    ext {
        springBootVersion = '2.1.1.RELEASE'
    }
    repositories {
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
    }
}

apply plugin: 'java'
apply plugin: 'war'
apply plugin: 'eclipse'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

group = 'jp.end0tknr'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = 1.8

repositories {
    mavenCentral()
}


dependencies {
    implementation('org.springframework.boot:spring-boot-starter-web')
    testImplementation('org.springframework.boot:spring-boot-starter-test')
    providedCompile('org.projectlombok:lombok:1.18.4')
}

mysqlのinformation_schema.tablesによるテーブルサイズの表示 - 改

ポイントは、data_free というデフラグ要な領域

InnoDBmysqlでは data_free という領域があり、 tableのデータを削除しても、ディスク容量は減らない。

ディスク容量を減らすには、postgresのvacuumや、hddのデフラグに該当する 1) 「OPTIMIZE TABLE $table_name;」または2)「ALTER TABLE $table_name ENGINE INNODB;」の実行が必要。

https://end0tknr.hateblo.jp/entry/20120729/1343513960

↑こちらの以前のエントリの再編集。

AWS RDSで表示されるディスク使用量と、information_schema.tables による使用量が 乖離していたことがきっかけ。

1)の実行には、事前にmysqlの設定が必要らしく、私の場合、2)を実行。

select  
table_name, engine, table_rows as tbl_rows, avg_row_length as rlen,  
floor((data_length+index_length+data_free)/1024/1024) as all_MB,
floor((data_length)/1024/1024)  as data_MB,
floor((index_length)/1024/1024) as index_MB,
floor((data_free)/1024/1024) as data_free_MB
from information_schema.tables  
where table_schema=database()  
order by (data_length+index_length) desc;
+-----------------------------+--------+----------+------+-------+-------+------+---------------+
| table_name                  | engine | tbl_rows | rlen | allMB | dMB   | iMB  | data_free_iMB |
+-----------------------------+--------+----------+------+-------+-------+------+---------------+
| csv_price                   | InnoDB | 27940304 |  511 | 15018 | 13624 | 1393 |           154 |
| csv_electricity             | InnoDB | 29248104 |  493 | 13765 | 13765 |    0 |          4888 |
| csv_battery_data            | InnoDB | 13357768 |  312 |  4391 |  3975 |  415 |             7 |
| csv_electricity_yesterday   | InnoDB |  1157275 | 3552 |  3921 |  3921 |    0 |         13200 |
| monthly_data                | InnoDB |  1555381 | 2538 |  3764 |  3764 |    0 |            12 |
| csv_price_yesterday         | InnoDB |   902017 | 4057 |  3551 |  3490 |   61 |          9505 |
| csv_electricity_hour        | InnoDB |  1386656 |  531 |   703 |   703 |    0 |             6 |
mysql> ALTER TABLE csv_price ENGINE INNODB;

と実行すると、↓この通り

select  
table_name, engine, table_rows as tbl_rows, avg_row_length as rlen,  
floor((data_length+index_length+data_free)/1024/1024) as all_MB,
floor((data_length)/1024/1024)  as data_MB,
floor((index_length)/1024/1024) as index_MB,
floor((data_free)/1024/1024) as data_free_MB
from information_schema.tables  
where table_schema=database()  
order by (data_length+index_length) desc;
+-----------------------------+--------+----------+------+-------+-------+------+---------------+
| table_name                  | engine | tbl_rows | rlen | allMB | dMB   | iMB  | data_free_iMB |
+-----------------------------+--------+----------+------+-------+-------+------+---------------+
| csv_price                   | InnoDB | 27940304 |  511 | 15018 | 13624 | 1393 |             0 |
| csv_electricity             | InnoDB | 29248104 |  493 | 13765 | 13765 |    0 |             0 |
| csv_battery_data            | InnoDB | 13357768 |  312 |  4391 |  3975 |  415 |             0 |
| csv_electricity_yesterday   | InnoDB |  1157275 | 3552 |  3921 |  3921 |    0 |             0 |
| monthly_data                | InnoDB |  1555381 | 2538 |  3764 |  3764 |    0 |             0 |
| csv_price_yesterday         | InnoDB |   902017 | 4057 |  3551 |  3490 |   61 |             0 |
| csv_electricity_hour        | InnoDB |  1386656 |  531 |   703 |   703 |    0 |             0 |

re: Javaビルドツール入門 Maven / Gradle / SBT / Bazel 対応

apache mavenの部分だけ読みましたが、非常に丁寧に書かれている。

何も考えず、まんま写経できる程。

Javaビルドツール入門 Maven/Gradle/SBT/Bazel対応 - 秀和システム あなたの学びをサポート

www.amazon.co.jp

SMB::Client for perl で接続し、file一覧をtraverse作成

↓こう書くみたい

#!/usr/local/bin/perl
use strict;
use warnings;
use Encode;
use File::Basename;
use File::stat;
use SMB::Client;
use Data::Dumper;

my $CONF =
    {nas_smb=>
     {share_uri=>'//10.100.21.???/nas',
      options=> {
                 username => 'ないしょ',
                 password => 'ないしょ',
                 quiet    => 0,
                 verbose  => 0,
                 log_level => SMB::LOG_LEVEL_INFO}},
    };


main( @ARGV );

sub main {
    my ($parent_path ) = @_;

    if(not defined($parent_path) or length($parent_path)==0){
        $parent_path = '';
    }
    
    # NAS接続client初期化
    my $smb_tree = init_nas_smb();
    unless($smb_tree){
        my $tmp_msg = "fail init_nas_smb()";
        die "$tmp_msg $!";
    }

    traverse_smb_tree($smb_tree, $parent_path);
}

sub traverse_smb_tree {
    my ($smb_tree, $parent_path ) = @_;

    my $find_path = join('/',$parent_path,'*');
    my @file_or_dirs = $smb_tree->find( $find_path );

    for my $file_or_dir ( @file_or_dirs ){
        
        next if( $file_or_dir->{name} eq '.' or
                 $file_or_dir->{name} eq '..');

        my $full_path = join('/', $parent_path, $file_or_dir->{name});

    # dir or fileの判定をいまいち、理解していません
        if( $file_or_dir->{attributes} == 16 ){
#        if( $file_or_dir->{allocation_size} == 0 ){
            print "DIR\t$full_path\n";
            traverse_smb_tree($smb_tree,$full_path);
            next;
        }

        my $disp_cols_str =
            join("\t",
                 "FILE",
                 $full_path,
                 $file_or_dir->size_string,
                 $file_or_dir->mtime_string);
        print $disp_cols_str,"\n";

    }
}

sub init_nas_smb {

    my $client = SMB::Client->new($CONF->{nas_smb}->{share_uri},
                                  %{$CONF->{nas_smb}->{options}});
    my $tree = $client->connect_tree;
    unless( $tree ){
        my $msg = "fail SMB::Client->connect_tree $CONF->{nas_smb}->{share_uri}";
        die "$msg $!";
    }
    return $tree;
}

1;

(再)リバースプロキシで認証したREMOTE_USERを受け側のapacheに渡す

https://qiita.com/end0tknr/items/c411cd91caaf43147ac4

↑こちらのurlを再編集。

前回は、リバースプロキシ側で認証した REMOTE_USER 情報を、 バックエンドのサーバに HTTP_X_FORWARDED_USER 情報とて渡しました。 今回は、バックエンドのサーバにも REMOTE_USER 情報として渡します。

構成

上段=前回、下段=今回で、【】の部分が変更箇所

┌Apache(REVERSE PROXY) ──┐  ┌Apache(APP SERVER)──────┐
│AuthType BASIC            ├→│AuthType NONE                 │
│【env param:REMOTE_USER】 │  │【env param:X-Forwarded-User】│
└─────────────┘  └───────────────┘
┌Apache(REVERSE PROXY) ──┐  ┌Apache(APP SERVER)──────┐
│AuthType BASIC            ├→│AuthType NONE                 │
│【env param:REMOTE_USER】 │  │【env param:REMOTE_USER】     │
└─────────────┘  └───────────────┘

apache httpd.confの編集(抜粋)

# この辺りのmoduleは必要
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule headers_module modules/mod_headers.so
LoadModule rewrite_module modules/mod_rewrite.so

# frontのreverse proxyにおけるbasic認証
<LocationMatch "/reverse_from/">
 AuthType Basic
 AuthName "Member Only"
 AuthUserFile /home/end0tknr/dev/htpasswd
 require valid-user
 ErrorDocument 401 /error/authen-error.html
</LocationMatch>

# frontのreverse proxy設定

# reverse proxyなので、off (通常のproxyならon)
ProxyRequests Off
<Location "/reverse_from/">
 ProxyPass         http://192.168.244.101:8080/reverse_to/
 ProxyPassReverse  http://192.168.244.101:8080/reverse_to/

 # remote user情報と X-Forwarded-User として、http headerに追加
 RewriteEngine On
 RewriteCond %{LA-U:REMOTE_USER} (.+)
 RewriteRule . - [E=RU:%1,NS]
 RequestHeader add X-Forwarded-User %{RU}e
</Location>


# backendのapp server設定
<Directory "/home/end0tknr/dev/reverse_to">
  Order allow,deny
  Allow from all

  # http headerのX-Forwarded-Userを環境変数=remote userに戻す
  RewriteEngine On
  RewriteCond %{REMOTE_USER} $^
  RewriteCond %{HTTP:X-Forwarded-User} (.+) [NC]
  RewriteRule . - [E=REMOTE_USER:%1,NS]

  # 変換結果をperl cgiで確認する為、CGI有効化
  <Files "*.pl">
    Options ExecCGI
    # この部分に「+」付きで、+FollowSymLinks を指定する必要があるみたい...
    Options +FollowSymLinks
    AddHandler cgi-script .pl
  </Files>
 </Directory>
Alias /reverse_to     /home/end0tknr/dev/reverse_to

apache の起動と確認

$ cd /home/end0tknr/local/apache24
$ ./bin/apachectl -f conf/httpd_reverse_proxy.conf 

と起動し、「ないしょIP:8080/reverse_from/index.pl 」へアクセスすると、 次のように表示されます。

CONTEXT_DOCUMENT_ROOT = /home/end0tknr/dev/reverse_to
CONTEXT_PREFIX = /reverse_to
DOCUMENT_ROOT = /home/end0tknr/local/apache24/htdocs
GATEWAY_INTERFACE = CGI/1.1
HTTP_ACCEPT = text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
HTTP_ACCEPT_ENCODING = gzip, deflate
HTTP_ACCEPT_LANGUAGE = ja-JP,ja;q=0.9,en-US;q=0.8,en;q=0.7
HTTP_CACHE_CONTROL = max-age=0
HTTP_CONNECTION = Keep-Alive
HTTP_COOKIE = _ga=GA1.1.994183502.1540097814; __utmz=224856154.1540122275.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utma=224856154.994183502.1540097814.1540329245.1540330341.5
HTTP_HOST = 192.168.244.101:8080
HTTP_UPGRADE_INSECURE_REQUESTS = 1
HTTP_USER_AGENT = Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
HTTP_X_FORWARDED_FOR = 192.168.244.1
HTTP_X_FORWARDED_HOST = 192.168.244.101:8080
HTTP_X_FORWARDED_SERVER = cent7.a5.jp
HTTP_X_FORWARDED_USER = endou021  ★★★
LD_LIBRARY_PATH = /home/end0tknr/local/apache24/lib
PATH = /usr/local/go/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/end0tknr/.local/bin:/home/end0tknr/bin
QUERY_STRING = 
REMOTE_ADDR = 192.168.244.101
REMOTE_PORT = 44614
REMOTE_USER = endou021  ★★★
REQUEST_METHOD = GET
REQUEST_SCHEME = http
REQUEST_URI = /reverse_to/index.pl
SCRIPT_FILENAME = /home/end0tknr/dev/reverse_to/index.pl
SCRIPT_NAME = /reverse_to/index.pl
SERVER_ADDR = 192.168.244.101
SERVER_ADMIN = you@example.com
SERVER_NAME = 192.168.244.101
SERVER_PORT = 8080
SERVER_PROTOCOL = HTTP/1.1
SERVER_SIGNATURE = 
SERVER_SOFTWARE = Apache/2.4.34 (Unix) PHP/7.2.6

print_env.plの内容は、以下。

#!/usr/local/bin/perl
use strict;
use warnings;
use CGI;
use Encode;
use Data::Dumper;

main();

sub main {
    my $q = CGI->new();
    print STDERR Dumper($q);

    print CGI::header(-type=>'text/plain',-charset=>'UTF-8');

    for my $env_key (sort keys %ENV){
        print "$env_key = $ENV{$env_key}\n";
    }
}

その他 - access_logの出力項目に X-Forwarded-For と X-Forwarded-User を追加

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T  %{X-Forwarded-For}i %{X-Forwarded-User}i" combined

httpd.conf のLogFormatを↑このように編集すれば、↓こう出力されます。

192.168.244.101 - - [23/Dec/2018:09:14:25 +0900] "GET /reverse_to/index.pl HTTP/1.1" 200 1620 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" 0  192.168.244.1 endou021
192.168.244.1 - endou021 [23/Dec/2018:09:14:25 +0900] "GET /reverse_from/index.pl HTTP/1.1" 200 1620 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" 0  - endou021

%{X-Forwarded-User}i は、%{REMOTE_USER}i と記載したかったのですが、その場合、上手く出力されませんでした。

go言語 ( golang ) オレオレ入門 - ほんの触りだけ

特徴 (自分の理解)

単一fileの実行可能file作成が容易

install

ソースからのGo言語インストール - golang.jp

上記urlのようにsrcからのinstallもありますが、今回は、downloadして解凍するだけ。

$ sudo su -
$ cd /usr/local
$ wget https://dl.google.com/go/go1.11.2.linux-amd64.tar.gz
$ tar -xvf go1.11.2.linux-amd64.tar.gz

$ vi /etc/profile

PATH=/usr/local/go/bin:$PATH  ## <-- L.53付近にadd
export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL

作業環境の準備 ( $GOHOME )

$GOHOME 以下にpackageのsrcや実行fileが集約されます。

$ vi ~/.bash_profile
export GOHOME=$HOME/go
#↑追加. package等のinstall先

emacsでのgo-mode

私の環境のemacs for winでは、package installできなかったので、 以下のように手作業でinstall

$ cd c:/emacs-24.5-IME-patched/share/emacs/site-lisp/
$ wget https://raw.githubusercontent.com/dominikh/go-mode.el/master/go-mode.el
$ vi ~/.emacs
(autoload 'go-mode "go-mode" nil t)
(add-to-list 'auto-mode-alist '("\\.go\\'" . go-mode))

packageの取得とbuild

以下の通り。

ただ、ローカル環境のbuildはどうするんでしょ? ちょっとしたcodeも GOHOME/src 以下に書くべきなんですかね?

# 「go get -d 」で $GOHOME/src以下に downloadのみ行います。
# 依存packageもdownloadする為、時間を要します。
# 「-d」がない場合、「go install」も行われます。
$ go get -d github.com/kahing/goofys

# 「go install」で実行fileが$GOHOME/src以下に作成されます。
$ go install github.com/kahing/goofys

# その他(参考)「go install」の場合、
# カレントディレクトリに作成されます。