Jenkins集成Gitlab时,测试Webhook时提示Webhook error: execution expired错误

之前gitlab中webhook的url按照Jenkins中提供的url设置(http://your.jenkins.url/project/projectName),然而在gitlab测试一直提示失败。在网上找了很多种办法无法解决,后更改url为http://your.jenkins.url/gitlab/build_now/projectName 成功触发

发表在 技术 | 留下评论

Springboot注入到Filter

在filter中加入如下代码

@Override
  public void init(FilterConfig filterConfig) throws ServletException {
    SpringBeanAutowiringSupport.processInjectionBasedOnServletContext(this,
        filterConfig.getServletContext());
  }

 

发表在 技术 | 留下评论

解决 (HTTP code 401) unexpected – {“message”:”Get https://registry-1.docker.io/v2/sath89/oracle-12c/manifests/latest: unauthorized: incorrect username or password”}

在使用kitematic下载oracle的docker image时,提示

 (HTTP code 401) unexpected - {"message":"Get https://registry-1.docker https://i....et/viagra-india/.io/v2/sath89/oracle-12c/manifests/latest: unauthorized: incorrect username or password"}

经过各种尝试后发现,是因为使用了email而非id登录kitematic导致的。registry-1.docker.io应该是根据用户id/密码去鉴权,而非email/密码,所以会导致鉴权失败。解决方法也很简单,就是改用id/密码登录

发表在 技术 | 标签为 , , | 一条评论

解决Nginx转发Websocket502错误

环境:前端通过Nginx转发Websocket请求到Tomcat,后端采用的Spring Websocket方式实现。

现象:用户连不上Websocket,查看开发者工具中的Network发现,Websocket链接状态都是502,并不断尝试重新链接。且该现象未找到任何规律,在某些用户浏览器中正常,某些情况下又不能访问。

定位:查看Nginx日志发现有“upstream prematurely closed connection while reading response header from upstream ”相关错误。查看Tomcat访问日志发现Websocket的请求全部返回了403,说明是没有权限访问。

解决:以“Websocket 403”为关键字Google后发现,应该是SpringWebsocket跨域默认拦截的问题,在WebsocketConfig中注册Websocket服务时加上

setAllowedOrigins("*")

即如下:

registry.addHandler(xxxHandler(), "/xxxHandler.do").addInterceptors(new xxxInterceptor()) https://yeapharmacy.com/f....setAllowedOrigins("*");

问题解决。

发表在 技术 | 留下评论

Java中几种遍历map的方式的性能比较

上一篇测试了list的几种遍历方式的性能,下边我们测试下map的遍历方式的性能。

首先map的遍历一般有以下几种:

  1. jdk8中新增的lambda遍历,map.forEach
  2. 遍历entrySet
  3. 遍历keySet,然后取出value

代码如下:

public static void main(String[] args) {
    Map<Integer,Integer> map= IntStream.range(0, 1000000).boxed().collect(Collectors.toMap((a)->a,(a)->a+1));
    int times = 10000;

    //lambda:66071(ms)
    long startTime = System.currentTimeMillis();
    for (int i = 0; i < times; i++) {
      lambda(map);
    }
    System.out.println("lambda:" + (System.currentTimeMillis() - startTime));

    //foreach_entry:91749
    startTime = System.currentTimeMillis();
    for (int i = 0; i < times; i++) {
      foreach_entry(map);
    }
    System.out.println("foreach_entry:" + (System.currentTimeMillis() - startTime));


    //foreach_key:123859
    startTime = System.currentTimeMillis();
    for (int i = 0; i < times; i++) {
      foreach_key(map);
    }
    System.out.println("foreach_key:" + (System.currentTimeMillis() - startTime));

  }

  private static void lambda(Map<Integer,Integer> map){
    int count[] = new int[]{0};
    map.forEach((k,v)->{
      if((k+v)%2==0){
        count[0] = count[0] + 1;
      }
    });
  }

  private static void foreach_entry(Map<Integer,Integer> map){
    int count[] = new int[]{0};
    for(Map espanolcialis.net.Entry<Integer,Integer> entry:map.entrySet()){
      if((entry.getKey()+entry.getValue())%2==0){
        count[0] = count[0] + 1;
      }
    }
  }

  private static void foreach_key(Map<Integer,Integer> map){
    int count[] = new int[]{0};
    for(Integer k:map.keySet()){
      if((k+map.get(k))%2==0){
        count[0] = count[0] + 1;
      }
    }
  }

可以看出lambda的方式性能最高,其次是遍历entrySet。

以上测试较为简单,并不能百分百的衡量几种遍历方式的真实性能,在实际环境中遇到过lamdba性能不如遍历entrySet的场景。所以在性能要求高的场景中最好实际测试下

发表在 技术 | 留下评论

Java中几种遍历List方式的性能比较

java中遍历List有以下几种方式:

  1. jdk8中新增的lambda遍历 (list.forEach)
  2. jdk5中的forEach
  3. 按下标遍历
  4. iterator迭代器

毫无疑问,最方便的方式就是采用lambda方式遍历,代码量最少,可是其性能如何呢?看下面实验。

硬件环境:MacBook Pro (Retina, 15-inch, Mid 2015) –

2.2 GHz Intel Core i7、

16 GB 1600 MHz DDR3

JDK版本:jdk1.8.0_91

代码如下

public static void main(String[] args) {
    List<Integer> in = IntStream.range(0, 1000000).boxed().collect(Collectors.toList());
    int times = 10000;

    //lambda:43124(ms)
    long startTime = System.currentTimeMillis();
    for (int i = 0; i < times; i++) {
      lambda(in);
    }
    System.out.println("lambda:" + (System.currentTimeMillis() - startTime));


    //foreach:29442
    startTime = System.currentTimeMillis();

    for (int i = 0; i < times; i++) {
      foreach(in);
    }
    System.out.println("foreach:" + (System.currentTimeMillis() - startTime));

    //for1:17517
    startTime = System.currentTimeMillis();

    for (int i = 0; i < times; i++) {
      for1(in);
    }
    System.out.println("for1:" + (System.currentTimeMillis() - startTime));

    //for2:18432
    startTime = System.currentTimeMillis();

    for (int i = 0; i < times; i++) {
      for2(in);
    }
    System.out.println("for2:" + (System https://ed-oesterreichische.at/viagra-generika/.currentTimeMillis() - startTime));


    //iterator:30997
    startTime = System.currentTimeMillis();

    for (int i = 0; i < times; i++) {
      iterator(in);
    }
    System.out.println("iterator:" + (System.currentTimeMillis() - startTime));
  }

  private static void foreach(List<Integer> in) {
    int count[] = new int[]{0};
    for (int a : in) {
      if (a % 2 == 0) {
        count[0] = count[0] + 1;
      }
    }

  }

  private static void lambda(List<Integer> in) {
    int count[] = new int[]{0};
    in.forEach(a -> {
      if (a % 2 == 0) {
        count[0] = count[0] + 1;
      }
    });

  }


  private static void for1(List<Integer> in) {
    int count[] = new int[]{0};
    for (int i = 0, j = in.size(); i < j; i++) {
      if (in.get(i) % 2 == 0) {
        count[0] = count[0] + 1;
      }
    }


  }


  private static void for2(List<Integer> in) {
    int count[] = new int[]{0};
    for (int i = 0; i < in.size(); i++) {
      if (in.get(i) % 2 == 0) {
        count[0] = count[0] + 1;
      }
    }

  }


  private static void iterator(List<Integer> in) {
    int count[] = new int[]{0};
    Iterator<Integer> it=in.iterator();
    while(it.hasNext()){
      if (it.next() % 2 == 0) {
        count[0] = count[0] + 1;
      }
    }

  }

对size为100万的list遍历1万次耗时对比,单位毫秒

由此图可以看出

  1. 耗时最短是传统的按下标循环遍历的模式,其次是foreach、iterator方式。耗时最长的是lambda方式。
  2. 循环调用size()方法会有一定的性能损失,可以将size赋给另一个变量

关于ArrayList遍历性能,在api文档中找到如下解释

public interface RandomAccess
Marker interface used by List implementations to indicate that they support fast (generally constant time) random access. The primary purpose of this interface is to allow generic algorithms to alter their behavior to provide good performance when applied to either random or sequential access lists.
The best algorithms for manipulating random access lists (such as ArrayList) can produce quadratic behavior when applied to sequential access lists (such as LinkedList). Generic list algorithms are encouraged to check whether the given list is an instanceof this interface before applying an algorithm that would provide poor performance if it were applied to a sequential access list, and to alter their behavior if necessary to guarantee acceptable performance.
It is recognized that the distinction between random and sequential access is often fuzzy. For example, some List implementations provide asymptotically linear access times if they get huge, but constant access times in practice. Such a List implementation should generally implement this interface. As a rule of thumb, a List implementation should implement this interface if, for typical instances of the class, this loop:
       for (int i=0, n=list.size(); i < n; i++)
           list.get(i);
   
runs faster than this loop:
       for (Iterator i=list.iterator(); i.hasNext(); )
           i.next();

也就是按下标遍历的方式的性能要由于迭代的方式。foreach循环的底层实现原理就是迭代器Iterator,参见Java语法糖1:可变长度参数以及foreach循环原理。所以后半句”反过来,如果是顺序访问的,则使用Iterator会效率更高”的意思就是顺序访问的那些类实例,使用foreach循环去遍历。

发表在 技术 | 留下评论

Aerospike 图形化客户端

基于java swing 的Aerospike简单图形化客户端,提供了bin完整数据显示,多条件查询,rownum(默认展示100条),简单的delete操作。支持多tab,多connects支持

git https://github.com/xingya771/aerospikeHelper

 

发表在 技术, 未分类 | 留下评论

使用Aerospike中遇到的那些坑

Aerospike 数据库是一个键-值存储的高性能实时 NoSQL(灵活模式)数据库,以低延迟和高吞吐量而闻名。以下记录我在使用过程中的遇到的一些“坑”:

1)Set无法drop

这个之所以出现在第一位,并不是因为其重要性,只是因为由于Set无法drop,导致要重建表只能delete all record,从而有可能会导致幽灵数据的问题。

解决方法:

  1. Insert records (through Java Client) into a set with expiration set for each record
  2. after all the records are expired, use asinfo to set ‘set-delete’ flag to TRUE
  3. restart aerospike
  4. The set will be dropped

2)delete record 后出现幽灵数据。

场景:将以前存储在LevelDB中的数据迁移到Aerospike中,对象采用的直接存储的方式。导入一部分数据后发现某些对象会报无法序列化的异常,就把所有使用到的bean都加了serialVersionUID。

结果使用一段时间后发现报serialVersionUID不一致的异常,以为是Aerospike的序列化BUg,就想不直接存储,改成存储JSON string。通过scanAll的方式遍历set,delete了所有的record,删除成功后,在aql中查询记录数为0。

测试没问题,一切正常,然而高兴没多久,发现后台又有异常,是在取出数据然后转成对象时候,说不能讲对象转成String。通过aql查询后发现,set中有几条非json形式的数据,看入库日期竟然是清空表前的时期。

经google后发现在Aerospike的社区中有过相关的讨论。原来delete record时只是删除了索引,并没有真正的删除数据(就像是只是在内存中删除,并未实时的刷到硬盘中,造成数据的一致性不能保证),在Aerospike重启后会重新load数据,这样被删除了数据就会重新出现。

解决办法:

a)合理设置TTL时间

b)尽量不使用硬删除,使用软删除打标记的方式

c)在网上看到如下方法,但是并没有尝试,不保证效果:在配置文件当中添加cold-start-        empty true,重启节点,此时节点会认为本地硬盘是空的,重新从集群中恢复数据,这          个时候我们删除的set就不会出现了

d)合理利用namespace

3)关于数据类型

a)最好不用直接使用boolean,到了数据库中会变成0/1

b)不要使用int/float数据类型,如果使用double类型,请保证小数点后一定有数字。之所以这么说,原因是int/float会变成long/double,而没有小位数的double会变成long

c)不要直接保存LinkedHashMap/TreeMap之类的map,因为保存时AS会强制转父类。

4)尽量不用使用单节点的部署方式,有很大可能性出现timeout。

具体表现为aql中可以正常show sets,但未set timeout时直接返回timeout,设置过后再次查询会导致查询hang住,无法取消、中断

 

发表在 技术, 未分类 | 留下评论

怎么让Aerospike中的SET显示PK?

Aerospike是一款性能强劲的NoSql数据库,以低延迟和高吞吐量而闻名。在SET(可以理解为表)中,一般来说PK作为主键,是不能被查询到的,但是有时候我们需要知道PK的值,这个如何解决呢?
看下com.aerospike.client.Key的源码,部分摘录如下:

/**
 * Unique record identifier. Records can be identified using a specified namespace,
 * an optional set name, and a user defined key which must be unique within a set.
 * Records can also be identified by namespace/digest which is the combination used 
 * on the server.
 */
public final class Key {
  /**
   * Namespace. Equivalent to database name.
   */
  public final String namespace;
  
  /**
   * Optional set name. Equivalent to database table.
   */
  public final String setName;
  
  /**
   * Unique server hash value generated from set name and user key.
   */
  public final byte[] digest;
  
  /**
   * Original user key. This key is immediately converted to a hash digest.
   * This key is not used or returned by the server by default. If the user key needs 
   * to persist on the server, use one of the following methods: 
   * <ul>
   * <li>Set "WritePolicy.sendKey" to true. In this case, the key will be sent to the server for storage on writes
   * and retrieved on multi-record scans and queries.</li>
   * <li>Explicitly store and retrieve the key in a bin.</li>
   * </ul>
   */
  public final Value userKey;
}

 

可以看到,Key有一个byte[] digest的数组对象,是根据set name 和key(PK)hash后的结果,但是这个是不可逆的。继续往下看能看到有个userKey,看解释是Original 的user key。但是默认它的值默认是null,除非显式的设置WritePolicy.sendKey=true,这样才会将PK显式的存储在Key中。

 

发表在 技术, 未分类 | 标签为 | 留下评论

解决SpringMVC中使用了WebSocket后通过JUnit测试报“No suitable default RequestUpgradeStrategy found”

具体异常信息如下:

java.lang.IllegalStateException: Failed to load ApplicationContext

  at org.springframework.test.context.CacheAwareContextLoaderDelegate.loadContext(CacheAwareContextLoaderDelegate.java:99)
  at org.springframework.test.context.DefaultTestContext.getApplicationContext(DefaultTestContext.java:101)
  at org.springframework.test.context.web.ServletTestExecutionListener.setUpRequestContextIfNecessary(ServletTestExecutionListener.java:155)
  at org.springframework.test.context.web.ServletTestExecutionListener.prepareTestInstance(ServletTestExecutionListener.java:100)
  at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:326)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.createTest(SpringJUnit4ClassRunner.java:212)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner$1.runReflectiveCall(SpringJUnit4ClassRunner.java:289)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.methodBlock(SpringJUnit4ClassRunner.java:291)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:232)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:89)
  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
  at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
  at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
  at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
  at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:175)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
  at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
  at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
  at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:253)
  at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'webSocketHandlerMapping' defined in class path resource [org/springframework/web/socket/config/annotation/DelegatingWebSocketConfiguration.class]: Instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public org.springframework.web.servlet.HandlerMapping org.springframework.web.socket.config.annotation.WebSocketConfigurationSupport.webSocketHandlerMapping()] threw exception; nested exception is java.lang.IllegalStateException: No suitable default RequestUpgradeStrategy found
  at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:592)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1094)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:989)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
  at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
  at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:304)
  at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
  at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:300)
  at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:195)
  at org plus d'information.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:700)
  at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
  at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
  at org.springframework.test.context.web.AbstractGenericWebContextLoader.loadContext(AbstractGenericWebContextLoader.java:129)
  at org.springframework.test.context.web.AbstractGenericWebContextLoader.loadContext(AbstractGenericWebContextLoader.java:60)
  at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.delegateLoading(AbstractDelegatingSmartContextLoader.java:100)
  at org.springframework.test.context.support.AbstractDelegatingSmartContextLoader.loadContext(AbstractDelegatingSmartContextLoader.java:250)
  at org.springframework.test.context.CacheAwareContextLoaderDelegate.loadContextInternal(CacheAwareContextLoaderDelegate.java:64)
  at org.springframework.test.context.CacheAwareContextLoaderDelegate.loadContext(CacheAwareContextLoaderDelegate.java:91)
  ... 29 more
Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Factory method [public org.springframework.web.servlet.HandlerMapping org.springframework.web.socket.config.annotation.WebSocketConfigurationSupport.webSocketHandlerMapping()] threw exception; nested exception is java.lang.IllegalStateException: No suitable default RequestUpgradeStrategy found
  at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:188)
  at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:581)
  ... 46 more
Caused by: java.lang.IllegalStateException: No suitable default RequestUpgradeStrategy found
  at org.springframework.web.socket.server.support.DefaultHandshakeHandler.initRequestUpgradeStrategy(DefaultHandshakeHandler.java:100)
  at org.springframework.web.socket.server.support.DefaultHandshakeHandler.<init>(DefaultHandshakeHandler.java:85)
  at org.springframework.web.socket.config.annotation.AbstractWebSocketHandlerRegistration.getOrCreateHandshakeHandler(AbstractWebSocketHandlerRegistration.java:131)
  at org.springframework.web.socket.config.annotation.AbstractWebSocketHandlerRegistration.getMappings(AbstractWebSocketHandlerRegistration.java:119)
  at org.springframework.web.socket.config.annotation.ServletWebSocketHandlerRegistry.getHandlerMapping(ServletWebSocketHandlerRegistry.java:67)
  at org.springframework.web.socket.config.annotation.WebSocketConfigurationSupport.webSocketHandlerMapping(WebSocketConfigurationSupport.java:36)
  at org.springframework.web.socket.config.annotation.DelegatingWebSocketConfiguration$$EnhancerByCGLIB$$b942a70d.CGLIB$webSocketHandlerMapping$2(<generated>)
  at org.springframework.web.socket.config.annotation.DelegatingWebSocketConfiguration$$EnhancerByCGLIB$$b942a70d$$FastClassByCGLIB$$ae93c7a7.invoke(<generated>)
  at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228)
  at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:326)
  at org.springframework.web.socket.config.annotation.DelegatingWebSocketConfiguration$$EnhancerByCGLIB$$b942a70d.webSocketHandlerMapping(<generated>)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:166)
  ... 47 more

而正常的运行却没有问题。

通过搜索发现是缺少Jetty Websocket-server的jar包。添加最新版本后,发现异常依旧,后发现最新版本(9.3.X)以后版本不支持Spring4.2以下版本。最后添加如下jar后解决:

    <dependency>
      <groupId>org.eclipse.jetty.websocket</groupId>
      <artifactId>websocket-server</artifactId>
      <version>9.2.17.v20160517</version>
      <scope>test</scope>
    </dependency>

 

发表在 技术, 未分类 | 2条评论