微博热搜API ｜高性能热搜API及基于Ak、Sk鉴权方案设计思路与实现

热搜API接口

上篇文章我们在《程序员盒子技术学院》社区分享了Java+jsonp实现微博热搜数据爬虫的方案，这篇文章分享下高性能api方案+开放api接口aksk鉴权方案，建议先学习上篇文章：

https://www.coderutil.com/star/exp/85

免费的微博热搜接口，为了保障接口性能，热搜数据每10分钟刷新一次，有需要的app可以接口，当前依然是免费接入的。

接口文档地址： https://www.coderutil.com/apiopen?tab=hotlist&pk=2000

两级缓存API性能保证

其实做前几天的文章中介绍过程序员盒子使用的多集缓存方案，没有看多的兄弟可以先了解下这篇文章（因为这里我们就不展开讲了，直接show me code）：

性能优化｜前端LocalStorage + Google Cache + Redis三级缓存性能优化

其实服务端接口这里还是用的Google cache+ redis实现两级缓存，这里并没有入库，完全依赖redis：

接口定义

/**

     * 微博热搜

     * @return

     */

    @GetMapping("/api/resou/v1/weibo")

    public APIResponseBean weiboHotSearch(@RequestParam(value = "size", defaultValue = "10") Integer size) {

        List<WeiboHotSearchResultBaseVO> result = hotSearchService.getWeiboHotSearchListFromCache(size);

        return APIResponseBeanUtil.success(result);

    }

针对跨域请求问题，我们还提供了jsonp跨域接口

/**

     * 百度热搜

     * @return

     */

    @RequestMapping(value = "/api/resou/v1/weibo.jsonp", produces = "text/script;charset=UTF-8", method= RequestMethod.GET)

    public String weiboHotSearchJsonp(HttpServletRequest request, String callback) {

        String sizeParam = request.getParameter("size");

        int size = StringUtils.isBlank(sizeParam) ? DEFAULT_SIZE : Integer.valueOf(sizeParam);

        List<WeiboHotSearchResultBaseVO> result = hotSearchService.getWeiboHotSearchListFromCache(size);

        return callback + "(" + JsonUtil.toJsonString(result) + ")";

    }

查询方法

 @Autowired

    private RedisService redisService;

    @Autowired

    private ApiUrlConfig apiUrlConfig;



    private static final Cache<String, String> RESOU_LOCAL_CACHE;



    static {

        RESOU_LOCAL_CACHE = CacheBuilder.newBuilder()

                .softValues()

                .maximumSize(10L)

                .expireAfterWrite(300L, TimeUnit.SECONDS)

                .build();

    }



   /**

     * 微博热搜

     * @return

     */

    public List<WeiboHotSearchResultBaseVO> getWeiboHotSearchListFromCache(int size) {

        String localCacheKey = LocalCacheKey.WEIBO.name();

        String localCacheVal = RESOU_LOCAL_CACHE.getIfPresent(localCacheKey);

        List<WeiboHotSearchResultBaseVO> list;

        if (StringUtils.isNotBlank(localCacheVal)) {

            list = JsonUtil.fromJson(localCacheVal, new TypeReference<List<WeiboHotSearchResultBaseVO>>() {});

        } else {

            String key = RedisKeyEnum.WEIBO_HOT_SEARCH_LIST_CACHE.getKey();

            String val = redisService.get(key);

            if (StringUtils.isNotBlank(val)) {

                list = JsonUtil.fromJson(val, new TypeReference<List<WeiboHotSearchResultBaseVO>>() {});

            } else {

                // 上篇文章中有这个刷新方法的具体实现逻辑

                list = refreshAndGetWeiboHotSearchListCache();

            }

            // 本地缓存如果没有则刷新到本地缓存

            RESOU_LOCAL_CACHE.put(localCacheKey, JsonUtil.toJsonString(list));

        }

        if (CollectionUtils.isNotEmpty(list)) {

            list = list.subList(0, size > list.size() ? list.size() : size);

        }

        return list;

    }

ok,两集缓存实现接口响应性能到底怎么样，我们看服务端日志稳定在10ms之内：

接口AKSK鉴权

开放z z鉴权Aksk是做常见的做法之一，这里我们其实实现的也比较简单，具体方案如下图所示：

（1）用户请求接口，header头邀请携带自己的app_key、secret_key

（2)APIFilter拦截API请求，做ak、sk做验证

（3）aksk验证通过请求api，请求缓存获取数据

（4）返回数据

APIFilter定义

@Slf4j

@Order(1)

@WebFilter(filterName = "apiRequestFilter", urlPatterns = {"/api/poster/*", "/api/resou/*", "/api/url/*",

        "/api/upload/*", "/api/text/*", "/api/ip/*", "/api/music/*", "/api/yulu/*", "/api/openai/*", "/rmi/*"})

public class ApiRequestFilter implements Filter {



    @Autowired

    private ApiInvokeRecordService apiInvokeRecordService;



    @Autowired

    private AccessSecretKeyService accessSecretKeyService;



    /***

     * 开放API清单

     */

    private static Map<String, String> openApiMap = new ConcurrentHashMap<>();



    private static List<String> WHITE_API_LIST = new ArrayList<>();



    static {

        

        // 需要Ak、Sk鉴权的请求

        openApiMap.put("/api/resou/v1/weibo", "微博热搜");



        /**

         * 白名单

         */

        WHITE_API_LIST.add("/api/poster/qr.temp");





    }



    @Override

    public void init(FilterConfig filterConfig) throws ServletException {



    }



    @Override

    public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {

        HttpServletRequest request = (HttpServletRequest) servletRequest;

        HttpServletResponse response = (HttpServletResponse) servletResponse;

        String uri = request.getRequestURI();

        if (isWhiteApiUri(uri)) {

            // 跳过鉴权

            filterChain.doFilter(servletRequest, servletResponse);

            return;

        }

        if (uri != null && uri.startsWith("rmi")) {

            String accessKey = request.getHeader("access-key");

            String secretKey = request.getHeader("secret-key");

            if (StringUtils.isBlank(accessKey) || StringUtils.isBlank(secretKey)) {

                APIResponseBean apiResponse = APIResponseBeanUtil.error(401, "无权限！");

                backErrorMessage(response, apiResponse);

                return;

            }

        }

        if (isApiUri(uri)) {

            // 获取API请求的认证参数：accessKey secretKey

            String accessKey = request.getHeader("access-key");

            String secretKey = request.getHeader("secret-key");

            // AK、SK 鉴权（走缓存查询）

            APIResponseBean apiResponseBean = accessSecretKeyService.checkRequestAkSk(accessKey, secretKey);

            if (!apiResponseBean.getSuccess()) {

                log.error("ApiRequestFilter.checkRequestAkSk error, AK:{}, SK:{}, uri:{}", accessKey, secretKey, uri);

                backErrorMessage(response, apiResponseBean);

                return;

            }

            if (StringUtils.isNotBlank(openApiMap.get(uri))) {

                // API调用埋点

                apiInvokeRecordService.point(uri, openApiMap.get(uri), accessKey);

            }

        }

        filterChain.doFilter(servletRequest, servletResponse);

    }



    @Override

    public void destroy() {



    }



    /***

     * 是否需要鉴权API

     * @param uri

     * @return

     */

    private boolean isApiUri(String uri) {

        if (openApiMap.keySet().contains(uri)) {

            return true;

        }

        for (String api : openApiMap.keySet()) {

            if (uri.startsWith(api)) {

                return true;

            }

        }

        return false;

    }



    /**

     * 是否需要鉴权API

     * @param uri

     * @return

     */

    private boolean isRmiApiUri(String uri) {

        if (openApiMap.keySet().contains(uri)) {

            return true;

        }

        for (String api : openApiMap.keySet()) {

            if (uri.startsWith(api)) {

                return true;

            }

        }

        return false;

    }



    /**

     * 是否白名单API

     * @param uri

     * @return

     */

    private boolean isWhiteApiUri(String uri) {

        return WHITE_API_LIST.contains(uri);

    }



    private void backErrorMessage(HttpServletResponse response, APIResponseBean apiResponseBean) {

        response.setContentType("application/json; charset=UTF-8");

        try {

            response.getWriter().print(JsonUtil.toJsonString(apiResponseBean));

        } catch (IOException e) {

            e.printStackTrace();

        }

    }

}